This repository provides several packages to support manipulation of a
content-addressable blob store.
The store package provides a Store type that provides a basic interface to
a blob store using a pluggable underlying storage representation, defined by
the storage package. The latter package comes equipped with two concrete
implementations, one using the local filesystem (via the standard os
package), the other using an in-memory map of strings. Any compatible
implementation of the storage.Interface will be compatible with the store
package. The cmd/blobstore binary provides a simple interface to a
file-based store for demo and experimentation.
The splitter package provides a context-sensitive block-splitter based on a
rolling hash function, similar to the technique used by
LBFS or rsync. A
splitter allows a large blob to be divided into chunks that are somewhat
resilient against local changes in the data. The cmd/tblock tool
demonstrates the use of the block splitting algorithm, and allows experimenting
with various parameters to the splitting algorithm.