Database Transfer

Four converters move documents and blobs between Database and CommitDatabase stores, in every combination of the two.

Each converter takes pre-opened source and target handles — never filesystem paths — so a transfer works over both local and remote transports. Every direction returns a DatabaseTransferInfo reporting how many documents and blobs it moved.

When to use: reach for these when data must change store kind or be replicated, rather than be read or mutated in place.

See also

Commit Database — the CommitDatabase model, including the Flatten pattern that CommitDatabaseFlattener automates.

The four directions

From	To	Class / method	Blobs kept
Database	Database	DatabaseCopier — .copy(source, target)	All, including orphans — a faithful replica
Database	CommitDatabase	DatabaseToCommitDatabaseConverter — .convert(source, target, label)	Only those the documents reference
CommitDatabase	Database	CommitDatabaseToDatabaseConverter — .convert(source, commitId, target)	Only those the chosen commit references
CommitDatabase	CommitDatabase	CommitDatabaseFlattener — .flatten(source, commitId, target, label)	Only those the chosen commit references

The two converters that target a CommitDatabase write the source state as a single commit under an explicit label; the two that read from one take an explicit commitId and reconstruct that commit’s state. There is no implicit current commit.

Blob handling

Only DatabaseCopier preserves orphan blobs, because it is the one faithful 1:1 replica. The three converters that narrow — into a commit, into a flat state, or into a single-commit history — first collect the blobs the resulting state actually references and copy only those, dropping orphans on the Database side and superseded history blobs on the CommitDatabase side.

Blobs are streamed in 64 MiB chunks, so payloads larger than 2 GB transfer without failing.

Quick Start

const { CommitDatabaseFlattener, CommitDatabaseToDatabaseConverter } =
    require('@digitalsubstrate/dsviper');

// Collapse a chosen commit into a fresh single-commit CommitDatabase,
// discarding history and superseded blobs (the Flatten pattern).
const info = new CommitDatabaseFlattener().flatten(
    source, headCommitId, target, 'flattened baseline');
console.log(info.documents(), 'documents,', info.blobs(), 'blobs');

// Materialize one commit's state into a plain, history-free Database.
new CommitDatabaseToDatabaseConverter().convert(source, commitId, flatDb);

source, target and flatDb are already-open database handles; see Database and Commit Database for opening and creating each kind.

Note

documents() and blobs() are methods here, where the Python binding exposes them as attributes (info.documents). They are also missing from the binding’s TypeScript declarations, so they do not appear in the Reference section below.

Progress reporting

Pass a callback as the last argument to observe a long-running transfer. It receives the current action and a percentage:

const { DatabaseCopier } = require('@digitalsubstrate/dsviper');

new DatabaseCopier().copy(source, target,
    (action, percent) => console.log(`${action}: ${percent.toFixed(0)}%`));

Tip

The Python binding takes a StepperDelegate subclass overriding step() where Node takes a plain function. A first-class callback is the JS idiom, so the binding has no StepperDelegate class to expose.

Key classes

Class	Purpose	Example
DatabaseCopier	Copies a Database into another Database, faithfully
CommitDatabaseFlattener	Collapses one chosen commit into a fresh single-commit CommitDatabase
DatabaseToCommitDatabaseConverter	Brings a flat Database under version control
CommitDatabaseToDatabaseConverter	Materializes one chosen commit as a flat, history-free Database

The mirror of this page for the Python binding is Database Transfer.

Classes

Class	Description
CommitDatabaseFlattener	Collapses one chosen commit into a fresh single-commit CommitDatabase
CommitDatabaseToDatabaseConverter	Materializes one chosen commit as a flat, history-free Database
DatabaseCopier	Copies a Database into another Database, faithfully
DatabaseToCommitDatabaseConverter	Brings a flat Database under version control
DatabaseTransferInfo	Result of a database conversion: counts of copied documents and blobs

Reference

class CommitDatabaseFlattener()

Collapses one chosen commit into a fresh single-commit CommitDatabase.

History and superseded blobs are discarded; the state is kept.

exported from index.d

CommitDatabaseFlattener.flatten(source, commitId, target, label, progress)

Flatten the state of commitId from source into target, a CommitDatabase

holding a single commit labelled label (history discarded). progress is an optional StepperDelegate. Return a DatabaseTransferInfo.

Arguments:

• source (CommitDatabase)

• commitId (ValueCommitId)

• target (CommitDatabase)

• label (string)

• progress ((action: string, percent: number) => void | undefined)

Returns:

DatabaseTransferInfo

class CommitDatabaseToDatabaseConverter()

Materializes one chosen commit as a flat, history-free Database. The

commit is explicit — there is no current one — and only the blobs it references are carried over.

exported from index.d

CommitDatabaseToDatabaseConverter.convert(source, commitId, target, progress)

Materialize the state of commitId from source into target, a Database.

progress is an optional StepperDelegate. Return a DatabaseTransferInfo.

Arguments:

• source (CommitDatabase)

• commitId (ValueCommitId)

• target (Database)

• progress ((action: string, percent: number) => void | undefined)

Returns:

DatabaseTransferInfo

class DatabaseCopier()

Copies a Database into another Database, faithfully.

Every blob comes over, orphans included — the one transfer that does not narrow.

exported from index.d

DatabaseCopier.copy(source, target, progress)

Copy every document and blob of source into target, as a faithful replica

(including blobs that no document references). progress is an optional StepperDelegate. Return a DatabaseTransferInfo.

Arguments:

• source (Database)

• target (Database)

• progress ((action: string, percent: number) => void | undefined)

Returns:

DatabaseTransferInfo

class DatabaseToCommitDatabaseConverter()

Brings a flat Database under version control.

The whole source state becomes one commit, under a label you give. Only the blobs the documents reference are carried over.

exported from index.d

DatabaseToCommitDatabaseConverter.convert(source, target, label, progress)

Convert source into target, a CommitDatabase, as a single commit labelled

label. progress is an optional StepperDelegate. Return a DatabaseTransferInfo.

Arguments:

• source (Database)

• target (CommitDatabase)

• label (string)

• progress ((action: string, percent: number) => void | undefined)

Returns:

DatabaseTransferInfo

class DatabaseTransferInfo()

Result of a database conversion: counts of copied documents and blobs.

Note: Not directly instantiable.

exported from index.d

DatabaseTransferInfo.blobs()

Return the number of blobs copied.

Returns:

bigint

DatabaseTransferInfo.documents()

Return the number of documents copied.

Returns:

number