Binary Data (Blobs)¶

Blobs provide efficient binary data storage with typed layouts and zero-copy NumPy integration.

When to use: Use blobs for binary data like images, meshes, and raw buffers. A builder fills the bytes, build() seals them into a ValueBlob, and a reader reads them: BlobArray flat and binary, BlobView element by element, BlobPack region by region.

Quick Start¶

>>> from dsviper import (BlobLayout, BlobArray, BlobArrayBuilder, BlobPack,
...                      BlobPackBuilder, BlobPackDescriptor)
>>> import numpy as np

A blob’s bytes never change once it is made, so writing happens on a builder that owns them: here 100 positions of 3 floats each.

>>> layout = BlobLayout('float', 3)
>>> builder = BlobArrayBuilder(layout, 100)

Its buffer is writable and flat — 300 floats, not a 100x3 matrix. Reshape it to address elements:

>>> flat = np.asarray(builder)
>>> flat.shape
(300,)
>>> flat.reshape(100, 3)[0] = [1.0, 2.0, 3.0]

build() hands the bytes over, and the value reads back read-only:

>>> del flat
>>> positions = builder.build()
>>> np.frombuffer(positions, dtype=np.float32)[:3]
array([1., 2., 3.], dtype=float32)
>>> np.frombuffer(positions, dtype=np.float32).flags.writeable
False

A BlobPack holds several regions in one blob, and carries their table:

>>> desc = BlobPackDescriptor()
>>> desc.add_region('positions', BlobLayout('float', 3), 100)
>>> desc.add_region('normals', BlobLayout('float', 3), 100)
>>> desc.add_region('indices', BlobLayout('uint', 3), 50)

>>> pack_builder = BlobPackBuilder(desc)
>>> with pack_builder['positions'] as region:
...     np.asarray(region).reshape(100, 3)[0] = [1.0, 2.0, 3.0]
>>> mesh = BlobPack.from_blob(pack_builder.build())
>>> np.frombuffer(mesh['positions'], dtype=np.float32).shape
(300,)

Choosing the Right Class¶

Data Type	Class	Example
Raw bytes	`ValueBlob`	`ValueBlob(bytes_data)`
Typed array	`BlobArray`	`BlobArray(blob_layout, blob)`
Multiple regions	`BlobPack`	Mesh with pos/normals/indices
Bytes, before they are a value	`BlobArrayBuilder`, `BlobPackBuilder`	`BlobArrayBuilder(layout, count)` → `build()`
Database reference	`ValueBlobId`	SHA-1 hash reference
Large files (>2GB)	`BlobStream`	Streaming upload

Core Classes¶

`dsviper.BlobLayout`	Describes the layout of one element in a blob: a data type and a component count.
`dsviper.BlobArray`	Reads blob as a flat array of blob_layout.data_type.
`dsviper.BlobArrayBuilder`	Fills the bytes of one typed array, then seals them into a ValueBlob.
`dsviper.BlobPack`	Several memory regions packed into one blob, described by a BlobPackDescriptor.
`dsviper.BlobPackBuilder`	Fills the regions a BlobPackDescriptor describes, then seals them into a ValueBlob a BlobPack reads.
`dsviper.BlobPackDescriptor`	Describes the binary regions a BlobPack will hold.
`dsviper.BlobPackRegion`	One region of a BlobPack - its name, the layout of its elements, how many it holds, and where it sits in the pack's blob.

Blob I/O¶

`dsviper.BlobStream`	The handle on a blob being written to a database past 2 GB, in chunks.
`dsviper.BlobEncoder`	Builds a blob element by element: write(value) appends, end_encoding() returns the finished blob.
`dsviper.BlobEncoderLayout`	A BlobLayout together with the Viper types it reads as.
`dsviper.BlobView`	Interprets the bytes of a blob through its layout.
`dsviper.BlobData`	The bytes of one blob, with the blob_id and layout that go with them.

Database Integration¶

`dsviper.BlobGetting`	Retrieves blobs from a persistence layer.
`dsviper.BlobInfo`	What a store knows about one blob without reading it: its id, layout, size, SQLite rowid, and whether it is chunked (over 2 GB).
`dsviper.BlobStatistics`	How many blobs a store holds and how big they are: count, total, smallest, largest, in bytes.