Database Transfer¶

Four converters move documents and blobs between Database and CommitDatabase stores, in every combination of the two.

Each converter takes pre-opened source and target handles — never filesystem paths — so a transfer works over both local and remote transports. Every direction returns a DatabaseTransferInfo reporting how many documents and blobs it moved.

When to use: reach for these when data must change store kind or be replicated, rather than be read or mutated in place.

The four directions¶

From	To	Class / method	Blobs kept
`Database`	`Database`	`DatabaseCopier` — `.copy(source, target)`	All, including orphans — a faithful replica
`Database`	`CommitDatabase`	`CommitDatabaseFromDatabaseConverter` — `.convert(source, target, label)`	Only those the documents reference
`CommitDatabase`	`Database`	`CommitDatabaseToDatabaseConverter` — `.convert(source, commitId, target)`	Only those the chosen commit references
`CommitDatabase`	`CommitDatabase`	`CommitDatabaseFlattener` — `.flatten(source, commitId, target, label)`	Only those the chosen commit references

The two converters that target a CommitDatabase write the source state as a single commit under an explicit label; the two that read from one take an explicit commitId and reconstruct that commit’s state. There is no implicit current commit.

Blob handling¶

Only DatabaseCopier preserves orphan blobs, because it is the one faithful 1:1 replica. The three converters that narrow — into a commit, into a flat state, or into a single-commit history — first collect the blobs the resulting state actually references and copy only those, dropping orphans on the Database side and superseded history blobs on the CommitDatabase side.

Blobs are streamed in 64 MiB chunks, so payloads larger than 2 GB transfer without failing.

Quick Start¶

const { CommitDatabaseFlattener, CommitDatabaseToDatabaseConverter } =
    require('@digitalsubstrate/dsviper');

// Collapse a chosen commit into a fresh single-commit CommitDatabase,
// discarding history and superseded blobs (the Flatten pattern).
const info = new CommitDatabaseFlattener().flatten(
    source, headCommitId, target, 'flattened baseline');
console.log(info.documents(), 'documents,', info.blobs(), 'blobs');

// Materialize one commit's state into a plain, history-free Database.
new CommitDatabaseToDatabaseConverter().convert(source, commitId, flatDb);

source, target and flatDb are already-open database handles; see Database and Commit Database for opening and creating each kind.

Note

documents() and blobs() are methods here, where the Python binding exposes them as attributes (info.documents). They are also missing from the binding’s TypeScript declarations, so they do not appear in the Reference section below.

Progress reporting¶

Pass a callback as the last argument to observe a long-running transfer. It receives the current action and a percentage:

const { DatabaseCopier } = require('@digitalsubstrate/dsviper');

new DatabaseCopier().copy(source, target,
    (action, percent) => console.log(`${action}: ${percent.toFixed(0)}%`));

Tip

The Python binding takes a StepperDelegate subclass overriding step() where Node takes a plain function. A first-class callback is the JS idiom, so the binding has no StepperDelegate class to expose.

Key classes¶

Class	Purpose	Example
`DatabaseCopier`	Copies a Database into another Database, faithfully
`CommitDatabaseFlattener`	Collapses one chosen commit into a fresh single-commit CommitDatabase
`CommitDatabaseFromDatabaseConverter`	Brings a flat Database under version control
`CommitDatabaseToDatabaseConverter`	Materializes one chosen commit as a flat, history-free Database

The mirror of this page for the Python binding is Database Transfer.

Classes¶

Class	Description
`CommitDatabaseFlattener`	Collapses one chosen commit into a single commit in another CommitDatabase
`CommitDatabaseToDatabaseConverter`	Materializes one chosen commit as a flat, history-free Database
`DatabaseCopier`	Copies a Database into another Database, faithfully
`CommitDatabaseFromDatabaseConverter`	Brings a flat Database under version control
`DatabaseTransferInfo`	Result of a transfer between two databases: counts of documents and blobs transferred

Reference¶

class CommitDatabaseFlattener()¶

Collapses one chosen commit into a single commit in another CommitDatabase.

History and superseded blobs are discarded; the state is kept. It opens an exclusive transaction of its own.

The target is a database you have just created: these transfers write into an empty one, and nothing checks it for you. The commit it writes is a root: it has no parent, so it continues nothing the target may already hold.

A run that throws part-way does not undo what it has already written. Every blob is copied inside one exclusive transaction, committed before the first document: a failure among them leaves none behind, a failure after them leaves them all, referenced by no commit. The documents are held in memory until the single commit is written. commitId() answers undefined either way. Discard the target and start again.

exported from index.d

CommitDatabaseFlattener.commitId()¶

Return the commit flatten() made, or undefined before it ran.

Reading the target’s heads instead would race: the write is over, so another stream may have added one. This is the id the flattening wrote, held by the object that wrote it - which is why a second flatten() throws, leaving both databases untouched.

Returns:: ValueCommitId | undefined

CommitDatabaseFlattener.flatten(source, commitId, target, label, progress)¶

Write the state of commitId from source into target as one root commit: labelled label; the history is not carried. progress is an optional callback, called with the action and a 0..1 fraction; one that throws is ignored and the run carries on. Return a DatabaseTransferInfo.

Arguments:

source (CommitDatabase)
commitId (ValueCommitId)
target (CommitDatabase)
label (string)
progress (((action: string, percent: number) => void) | null)

Returns:

DatabaseTransferInfo

class CommitDatabaseToDatabaseConverter()¶

Materializes one chosen commit as a flat, history-free Database.

The commit is explicit - there is no current one - and only the blobs it references are carried over.

The target is a database you have just created: these transfers write into an empty one, and nothing checks it for you.

A run that throws part-way does not undo what it has already written. The blobs and the documents share one exclusive transaction that spans the whole conversion, so neither lands however far it got; the definitions do. Discard the target and start again.

exported from index.d

CommitDatabaseToDatabaseConverter.convert(source, commitId, target, progress)¶

Materialize the state of commitId from source into target.

target is a Database. progress is an optional callback, called with the action and a 0..1 fraction; one that throws is ignored and the run carries on. Return a DatabaseTransferInfo.

Requires that target have no transaction open, and throws when it has: the conversion opens an exclusive one of its own.

Arguments:

source (CommitDatabase)
commitId (ValueCommitId)
target (Database)
progress (((action: string, percent: number) => void) | null)

Returns:

DatabaseTransferInfo

class DatabaseCopier()¶

Copies a Database into another Database, faithfully.

Every blob comes over, orphans included, where a conversion carries only the blobs a document references.

The target is a database you have just created: these transfers write into an empty one, and nothing checks it for you.

exported from index.d

DatabaseCopier.copy(source, target, progress)¶

Copy every document and blob of source into target.

progress is an optional callback, called with the action and a 0..1 fraction; one that throws is ignored and the run carries on. Return a DatabaseTransferInfo.

Requires that target have no transaction open, and throws when it has: the copy opens an exclusive one of its own. source may be in a transaction.

Arguments:

source (Database)
target (Database)
progress (((action: string, percent: number) => void) | null)

Returns:

DatabaseTransferInfo

class CommitDatabaseFromDatabaseConverter()¶

Brings a flat Database under version control.

The whole source state becomes one commit, under a label you give. Only the blobs the documents reference are carried over.

The target is a database you have just created: these transfers write into an empty one, and nothing checks it for you. The commit it writes is a root: it has no parent, so it continues nothing the target may already hold.

A run that throws part-way does not undo what it has already written. Every blob is copied inside one exclusive transaction, committed before the first document: a failure among them leaves none behind, a failure after them leaves them all, referenced by no commit. The documents are held in memory until the single commit is written. commitId() answers undefined either way. Discard the target and start again.

exported from index.d

CommitDatabaseFromDatabaseConverter.commitId()¶

Return the commit convert() made, or undefined before it ran.

Reading the target’s heads instead would race: the write is over, so another stream may have added one. This is the id the conversion wrote, held by the object that wrote it - which is why a second convert() throws, leaving both databases untouched.

Returns:: ValueCommitId | undefined

CommitDatabaseFromDatabaseConverter.convert(source, target, label, progress)¶

Convert source into target, a CommitDatabase, as a single commit labelled: label. progress is an optional callback, called with the action and a 0..1 fraction; one that throws is ignored and the run carries on. Return a DatabaseTransferInfo.

Arguments:

source (Database)
target (CommitDatabase)
label (string)
progress (((action: string, percent: number) => void) | null)

Returns:

DatabaseTransferInfo

class DatabaseTransferInfo()¶

Result of a transfer between two databases: counts of documents and blobs transferred.

Note: Not directly instantiable.

exported from index.d

DatabaseTransferInfo.blobs()¶

Return the number of blobs transferred.

Returns:: number

DatabaseTransferInfo.documents()¶

Return the number of documents transferred.

Returns:: number