pub struct GearChunker {
pub min: usize,
pub avg: usize,
pub max: usize,
pub normalization_level: u8,
}Expand description
Gear hash rolling chunker configuration.
Fields§
§min: usize§avg: usize§max: usize§normalization_level: u8Normalization level for chunk size distribution.
- 0: Legacy single-mask (geometric distribution, backward compatible)
- 1: FastCDC normalized (dual-mask, tighter distribution around avg)
- 2: Aggressive normalization (very tight, may reduce dedup ratio)
§CAS Compatibility Warning
Changing this field produces different chunk boundaries and therefore different BLAKE3 chunk hashes. Existing CAS data will not deduplicate against chunks produced at a different normalization level.
Implementations§
Source§impl GearChunker
impl GearChunker
Sourcepub fn new(min: usize, avg: usize, max: usize) -> Result<Self>
pub fn new(min: usize, avg: usize, max: usize) -> Result<Self>
Create a new gear hash chunker with the given size bounds.
Sourcepub fn with_normalization(self, level: u8) -> Result<Self>
pub fn with_normalization(self, level: u8) -> Result<Self>
Set the normalization level for chunk size distribution.
Returns an error if level > 2 – higher levels shift masks beyond useful ranges.
Also rejects configurations where logarithm2(avg) <= level, which would produce
a zero mask_l (MASKS[0]) causing degenerate 1-byte chunks.
Sourcepub fn chunk_reader<R: Read>(&self, reader: R) -> Result<Vec<Chunk>>
👎Deprecated: Use StreamingChunker for files > 16 MB
pub fn chunk_reader<R: Read>(&self, reader: R) -> Result<Vec<Chunk>>
Use StreamingChunker for files > 16 MB
Chunk an entire reader, returning all chunks with their BLAKE3 hashes.
§Limits
Rejects inputs larger than 16 MB to prevent OOM. Use StreamingChunker
for arbitrarily large files.
Sourcepub fn find_boundaries(&self, data: &[u8]) -> Vec<usize>
pub fn find_boundaries(&self, data: &[u8]) -> Vec<usize>
Find chunk boundaries in a byte slice using gear hash.
When normalization_level == 0, uses a single mask (avg - 1) – backward-compatible
with all existing data. When > 0, uses FastCDC dual-mask: a stricter mask_s below avg
and a relaxed mask_l above avg, tightening the chunk-size distribution.
Sourcepub fn chunk_slice(&self, data: &[u8]) -> Vec<Chunk>
pub fn chunk_slice(&self, data: &[u8]) -> Vec<Chunk>
Chunk from a byte slice directly (avoids double-copy for in-memory data).
Trait Implementations§
Source§impl Debug for GearChunker
impl Debug for GearChunker
Auto Trait Implementations§
impl Freeze for GearChunker
impl RefUnwindSafe for GearChunker
impl Send for GearChunker
impl Sync for GearChunker
impl Unpin for GearChunker
impl UnsafeUnpin for GearChunker
impl UnwindSafe for GearChunker
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more