The Service Mechanism

A Viper::Service aggregates DSM definitions and function pools into an immutable container that can be exposed over a network socket. The Transparent Remote Access principle of the Viper runtime means clients use the same typed pool API whether they are calling local or remote functions — the network is hidden by the runtime.

Two pool types

A service exposes two kinds of typed pool, each with distinct execution semantics.

FunctionPool — stateless

Groups stateless functions that execute on the server and return a value. Pure functions: same inputs, same outputs, no access to storage or mutable state.

"""This pool provides utility functions."""
function_pool Tools {7aa5aea2-c9de-4f91-8371-7995aca8c947} {
    int64   add(int64 a, int64 b);
    Vector3 addVector(Vector3 a, Vector3 b);
    string  randomString(uint32 size);
};

• Stateless — no AttachmentMutating, no backing storage, no side effects on persisted data.

• Immediate — execute synchronously, return immediately.

• Type-safe — parameters and return value validated through the generated FunctionPrototype.

AttachmentFunctionPool — stateful

Groups functions that operate inside a stateful context, with access to an AttachmentMutating for reads and mutations.

"""This pool provides Player utility functions."""
attachment_function_pool PlayerModel {d75a8a57-...} {
    mutable key<Player> create(string nickname, Level level);
    optional<key<Player>> has_player(string nickname);
};

• Stateful — access to an AttachmentMutating passed by the client at call time.

• mutable — mutating methods explicitly marked in the DSM.

• AttachmentMutating is the protocol, not the storage. The service has no opinion on what backs the interface — the client picks the implementation that fits its scenario, and the service reads from / writes to it through the interface alone.

Server side — assembling and serving

A service binary is small: it composes pools into a Viper::Service, opens a socket, and runs Viper::ServiceServer.

The canonical worked example lives in the devkit-codegen-test/service/ repository, dedicated to exercising the Service codegen end to end. From service/ServiceServer.cpp:

// 1) Compose the service from definitions and pools.
auto const service = Viper::Service::make(
    Service::definitions(),
    {                                          // FunctionPools
        Service::FunctionPools::tools(),
    },
    {                                          // AttachmentFunctionPools
        Service::AttachmentFunctionPools::playerModel(),
    });

// 2) Open a server socket — TCP or Unix.
auto const socket = Viper::Socket::makePassiveInet("0.0.0.0", "54328");

// 3) Run the service (blocking).
Viper::ServiceServer::run(socket, service, logging);

Three things to note:

• Service::definitions() is the embedded DSM — generated by Kibo from Service.dsm.json, shipped inside the binary so that connecting clients can discover the typed surface at connect time without prior knowledge of the schema.

• Service::FunctionPools::tools() and Service::AttachmentFunctionPools::playerModel() are the Kibo-generated pool C++ classes. The hand-written bridge files (Service_FunctionPoolBridges.cpp, Service_AttachmentFunctionPoolBridges.cpp) connect the generated pool methods to the actual business logic.

• A service binary can also pair with a UI application by reusing the same pool instances the UI process composes — same business surface, no UI tier. The UI process and the service binary then share the pool C++ classes and the bridge code, and publish those pools through different surfaces (in-process API on one side, RPC on the other).

Client side — Dual Reality, two languages

The Python and C++ clients consume the same Viper RPC protocol through the Dual Reality pattern: an adapter between Viper’s dynamic, metadata-driven runtime and the static API world where developers and IDEs are most productive.

Two paths reach the same RPC layer:

• Dynamic — a generic universal client that introspects any Viper service at connect time and dispatches calls through a runtime dictionary. No per-service codegen, ever.

• Static — per-pool typed proxies generated by Kibo from the service’s DSM. Their only purpose is developer / IDE comfort — autocompletion, compile-time type checking, jump-to-definition. They wrap the same dynamic dispatch underneath.

The dynamic path — one universal generic client, zero per-service codegen

Viper::ServiceRemote::connect(...) returns a handle that exposes each pool’s functions as a runtime-introspected dictionary. This path works without importing any service-specific generated module — the same client connects to any Viper service:

# Pure dynamic call — no per-service codegen needed
from dsviper import Definitions, ServiceRemote

defs = Definitions()
s = ServiceRemote.connect("localhost", "54328", defs)

result = s.function_pool_funcs("Tools")["add"](32, 10)   # returns 42

The minimal connection scaffold ships with dsviper-tools as service_client.py — five lines from which any diagnostic, REPL exploration, or schema-agnostic adapter extends.

The static Python path — Kibo-generated typed proxies

Kibo emits per-service Python modules: function_pool_remotes.py (one class per function_pool) and attachment_function_pool_remotes.py (one class per attachment_function_pool). Each class is a thin typed wrapper over the dynamic function_pool_funcs(...) dispatch.

Sketch, derived from devkit-codegen-test/service/python/service_client.py:

from dsviper import Definitions, ServiceRemote
from service.data import *
import service.function_pool_remotes            as fpr
import service.attachment_function_pool_remotes as afpr

defs           = Definitions()
service_remote = ServiceRemote.connect("localhost", "54328", defs)

# Stateless — typed Python proxy from generated function_pool_remotes.
TOOLS = fpr.Tools(service_remote)
if TOOLS.is_available():
    r  = TOOLS.add(32, 10)                                     # 42
    vr = TOOLS.add_vector(Demo_Vector3({"x":1, "y":2, "z":3}),
                          Demo_Vector3({"x":10,"y":20,"z":30}))

# Stateful — typed Python proxy from
# attachment_function_pool_remotes. The client passes whatever
# AttachmentMutating implementation it wants to use as the receptacle
# (built from any backend — the service does not know which).
PM = afpr.PlayerModel(service_remote)
if PM.is_available():
    mutating = ...   # any AttachmentMutating implementation
    # Execution runs on the SERVER — mutations land in the
    # CLIENT-LOCAL `mutating` via the Remote-Local protocol below.
    key = PM.create(mutating, "the shadow man", Demo_Level.BEGINNER)

The static C++ path — Kibo-generated typed proxies

Symmetric to Python, with per-pool C++ headers Service_FunctionPoolRemotes.hpp and Service_AttachmentFunctionPoolRemotes.hpp. From devkit-codegen-test/service/ServiceClient.cpp:

#include "Service_FunctionPoolRemotes.hpp"
#include "Service_AttachmentFunctionPoolRemotes.hpp"

using namespace Service::FunctionPoolRemotes;
using namespace Service::AttachmentFunctionPoolRemotes;

auto const definitions = Viper::Definitions::make();
auto const service     = Viper::ServiceRemote::connect(
    inetAddress, inetPort, definitions);

// Stateless call — typed C++ proxy.
auto const tools = Tools{service};
if (tools.isAvailable()) {
    auto const r = tools.add(32, 10);                          // 42
}

// Stateful call — typed C++ proxy. The client passes its own
// AttachmentMutating receptacle (from any backend implementing
// the interface) as the call's state.
auto const playerModel = PlayerModel{service};
if (playerModel.isAvailable()) {
    std::shared_ptr<Viper::AttachmentMutating> mutating = /* … */;

    // Execution runs on the SERVER — mutations land in the
    // CLIENT-LOCAL `mutating` via the Remote-Local protocol below.
    auto key = playerModel.create(mutating, "the shadow man",
                                  Demo::Level::Beginner);
}

service->close();

The Kibo-generated parts are the per-pool proxy classes (Tools, PlayerModel in both languages) and the marshalling for each method. Both languages get them from the same DSM, so the same pool name has the same method signatures on both sides.

The Remote-Local protocol — how stateful pools stay safe over RPC

For AttachmentFunctionPool calls, the protocol implements a pattern that is unusual but central to the design: business logic executes on the server, but state mutations happen on the client. The roles invert during execution.

When a stateful pool method runs server-side and needs to read or mutate the client’s AttachmentMutating, the server sends a callback RPC back to the client. The client reads (or mutates) its client-local AttachmentMutating, responds, and the server continues.

CLIENT                                         SERVER
   │                                              │
   │──► RPCPacketCallAttachmentFunction ─────────►│
   │    (pool: PlayerModel, func: create)         │
   │                                              │
   │                                        ┌─────┴─────┐
   │                                        │  Execute  │
   │                                        │  business │
   │                                        │   logic   │
   │                                        └─────┬─────┘
   │                                              │
   │◄── RPCPacketCallAttachmentGettingGet ◄───────│  ← ROLE INVERSION
   │    (server asks client to read)              │    Server CALLS
   │                                              │
   │    [Read from LOCAL AttachmentMutating]      │
   │──► RPCPacketReturnValue ────────────────────►│  ← Client RESPONDS
   │                                              │
   │◄── RPCPacketCallAttachmentMutatingUpdate ◄───│  ← ROLE INVERSION
   │    (server asks client to mutate)            │
   │                                              │
   │    [Mutate CLIENT-LOCAL AttachmentMutating]  │
   │──► RPCPacketReturnVoid ─────────────────────►│
   │                                              │
   │                                        [Function complete]
   │                                              │
   │◄── RPCPacketReturnValue (final result) ◄─────│
   │                                              │

On the server, the mutating parameter passed to the pool function is an AttachmentMutatingRemote — a proxy that forwards every operation to the client. The bridge code that implements the pool method does not know it is talking to a remote state:

// Server-side bridge — the same code runs locally and remotely.
// When called via RPC, `mutating` is AttachmentMutatingRemote,
// which proxies every Set/Get/Update back to the client.
PlayerKey create(std::shared_ptr<Viper::AttachmentMutating> const & mutating,
                 std::string const & nickname, Demo::Level level) {
    auto const key      = PlayerKey::create();
    auto const property = PlayerProperty{nickname, level};
    Attachments::Player_Property::set(mutating, key, property);  // proxied!
    return key;
}

The Viper RPC layer defines fifteen callback packet types covering every read and mutation operation an AttachmentMutating can perform — three reads (Keys, Has, Get) and twelve mutations (Set, Diff, Update on documents; UnionInSet, SubtractInSet on sets; map and ordered-array mutations).

Safety by isolation

The Remote-Local pattern looks dangerous at first read — the server sends commands that mutate client state. The design is inherently safe because mutations land in the client’s AttachmentMutating, and persistence (if any) is triggered by the client, never by the server.

CLIENT                                         SERVER
   │                                              │
   │  ┌─────────────────────────┐                 │
   │  │  AttachmentMutating     │                 │
   │  │  (CLIENT-LOCAL)         │                 │
   │  │                         │                 │
   │  │  All mutations land     │◄────────────────│  Server sends mutations
   │  │  here, locally          │                 │
   │  └───────────┬─────────────┘                 │
   │              │                               │
   │              │ Persistence (if any) is       │
   │              │ TRIGGERED BY THE CLIENT       │
   │              │ — never by the server.        │
   │              ▼                               │
   │  ┌─────────────────────────┐                 │
   │  │  Backing store          │                 │
   │  │  (whatever the client   │                 │
   │  │   chose to bind to its  │                 │
   │  │   AttachmentMutating)   │                 │
   │  └─────────────────────────┘                 │

Failure mode	Result	Data corruption
Network timeout	Client-local state abandoned	Impossible
Server exception	Client-local state abandoned	Impossible
Client crash before persist	Client-local state lost	Impossible
Server sends invalid data	Client validates before persisting	Impossible
Partial mutations	Never reach the backing store	Impossible

When to use a service

Add a service tier when one of these is true:

• The business surface should be reachable from outside its own process — multiple clients, scripting from another tool, integration with an unrelated language.

• Heavy computation must run on a powerful server, not on a resource-constrained client.

• The same business logic must back several user-facing surfaces (desktop app, web app, CLI tool) without duplication.

A service is not useful when:

• The work is single-user and single-process — calling pools in-process is sufficient and faster.

• The exchange is purely about transporting an opaque database artefact across clients with no domain-specific operations to expose. For that, dedicated database-transport tools (e.g. the commit_database_server for versioned databases) are simpler.

References

Source code

Location	What
devkit-codegen-test/service/	Canonical worked example — DSM, generate.py, server, two clients, bridges
devkit-codegen-test/service/ServiceServer.cpp	Server binary
devkit-codegen-test/service/ServiceClient.cpp	C++ client using generated typed proxies
devkit-codegen-test/service/python/service_client.py	Python client using generated typed proxies
dsviper-tools/service_client.py	Five-line connection scaffold for the universal dynamic client

dsviper API

• dsviper.ServiceRemote, dsviper.ServiceRemoteFunction, dsviper.ServiceRemoteAttachmentFunction, dsviper.ServiceRemoteAttachmentFunctionPoolFunction — the client-side surface (see dsviper API).

DSM language

• function_pool and attachment_function_pool keywords — see Concepts and Hierarchies.

Related

• Database Server — the commit database server, a different mechanism (transports a CommitDatabase artefact, does not invoke function pools).