Binary formats, compression and numerical work

Packed bytes, explicit numeric widths and independent-value collections provide the foundation for binary protocols and scientific utilities. Keep representation choices explicit: a byte offset is different from a Unicode position, and a machine word is not automatically an external wire encoding.

Read fixed-width byte values

DIM bytes AS LIST OF BYTE = [BYTE(1), BYTE(2), BYTE(3), BYTE(4)]
PRINT ReadUInt16LE(bytes, 0)
PRINT ReadUInt32LE(bytes, 0)

These little-endian readers accept arbitrary byte alignment and validate logical length. Invalid offsets or insufficient bytes report IndexError carrying the original offset. ReadUInt16LE/32LE/64LE return UINT32/UINT32/UINT64 respectively. They read an external representation without exposing raw pointers.

Unsigned types have checked ordinary arithmetic. Named wrapping, shifting, rotating, wide multiplication and layout helpers serve operations that deliberately need machine-width behavior. Use the unsigned reference for exact names and range rules. Do not rely on signed overflow to implement a binary algorithm.

Compress a byte buffer

The import path below is correct for a source saved in the repository root:

IMPORT "stdlib/compress/gzip.bas" AS gzip
original = EncodeUtf8("Hello, BASIC!")
packed = gzip.Compress(original)
restored = gzip.Decompress(packed, MaxOutput=1024)
PRINT DecodeUtf8(restored)

The imported codecs operate on LIST OF BYTE. DEFLATE, zlib, gzip and LZW encode and decode; bzip2 is decoding-only. LZW selects bit order and literal width. gzip supports checked metadata and individual member decoding. Compression levels choose effort, not identical output to every other implementation.

Always choose an appropriate MaxOutput for untrusted compressed data. The limit counts actual decoded bytes, including concatenated members. Header/checksum/size errors use CompressionError rather than returning a partly successful buffer. Preset dictionaries and reusable streaming codec objects are not exposed yet.

Hash bytes

IMPORT "stdlib/sha256.bas" AS sha256
PRINT sha256.HexDigest(EncodeUtf8("abc"))

SHA-256 Digest returns 32 bytes; HexDigest returns lowercase hexadecimal text. This is a digest facility, not password hashing, authentication or a complete cryptographic suite. A collection Hash is a separate operation with a different purpose; it is not a cryptographic digest.

Use numerical vectors

The scalar math library includes roots/powers, exponential/logarithmic functions, trigonometry in radians, angle conversion, rounding, fused multiply-add, binary decomposition/scaling and adjacent-value operations. POW performs exponentiation; there is no ^ power operator. FLOOR/INT return a DOUBLE floor, whereas CINT truncates to INTEGER and checks range. ROUND uses nearest rounding with ties to even. FMOD and REMAINDER use different quotient rules. The included math reference lists every family and its special values.

a = [1.0, 2.0, 3.0]
b = [4.0, 5.0, 6.0]
PRINT Dot(a, b)
PRINT Norm([3.0, 4.0])
PRINT FloorDiv(-17, 16)
PRINT EuclideanMod(-17, 16)

Dot, Norm, Normalized, Distance, CosineSimilarity, Scaled and component-wise VectorAdd/Subtract/Multiply/Divide accept supported numeric lists and rank-one arrays. Inputs borrow; returned vectors own new list values. Shape mismatches and invalid zero norms are checked. Integral operations retain width/range checks; floating operations retain IEEE NaN/infinity behavior.

For coordinates, FloorDiv and EuclideanMod give floor-based quotient/nonnegative remainder semantics, unlike ordinary truncating integer division. FloorToInteger explicitly converts a floating floor. Use the vector and floating references for empty inputs, precision, nonfinite values and scheduling behavior.

Accumulate bytes and text efficiently

ByteBuffer stores initialized bytes separately from reserved capacity. StringBuilder does the same for valid text. Reserve is a performance hint, not a length change. Clear retains capacity; TakeBytes/TakeString extract the accumulated value and empty the builder. ToBytes/ToString take independent snapshots instead.

builder = StringBuilder(Capacity=128)
Append(BYREF builder, "Hello, ")
Append(BYREF builder, "λ")
PRINT Length(builder)
PRINT ByteCount(builder)
PRINT TakeString(BYREF builder)
PRINT Length(builder)
buffer = ByteBuffer(Capacity=16)
Extend(BYREF buffer, EncodeUtf8("AB"))
snapshot = ToBytes(buffer)
WriteAt(BYREF buffer, 0, [BYTE(90)])
PRINT DecodeUtf8(snapshot)
PRINT DecodeUtf8(Bytes(buffer))

Bytes(buffer) can lend a read-only view directly to a consumer. That loan keeps the source alive and prevents conflicting mutation or transfer during the call. For example, extending a buffer from its own borrowed view conflicts; take a snapshot first. Builders themselves are copyable independent values. Byte offsets are nonnegative initialized positions, not Python-style negative string indexes. WriteAt replaces existing bytes without extending the buffer; Truncate only shrinks. The buffer reference covers AppendChar, AppendLine, spans and capacity operations.

Read and write binary fields with a cursor

ByteWriter removes manual offset arithmetic when assembling a packet. ByteReader tracks a checked position while parsing it. The default byte order is little endian; select big endian for protocols that require it.

writer = ByteWriter(Capacity=32)
WriteUInt32(BYREF writer, UINT32(42), Order=ByteOrder.BigEndian)
WriteInt16(BYREF writer, -7)
WriteBytes(BYREF writer, EncodeUtf8("OK"))
bytes = TakeBytes(BYREF writer)
reader = ByteReader(TAKE bytes)
PRINT ReadUInt32(BYREF reader, Order=ByteOrder.BigEndian)
PRINT ReadInt16(BYREF reader)
PRINT DecodeUtf8(ReadBytes(BYREF reader, 2))
PRINT Remaining(reader)
TRY
    value = ReadByte(BYREF reader)
CATCH err AS IndexError
    PRINT Position(reader)
END TRY

Readers snapshot ordinary input; TAKE adopts a named byte list. Copying a reader forks its position and contents. Seek and Skip are checked. A failed field read leaves the position unchanged; ReadInto also validates its full destination range before mutation. Cursor ReadByte requires a byte and fails at EOF, unlike the OPTIONAL byte returned by a buffered stream. Signed 8/16/32/64, unsigned 16/32/64 and exact binary64 double fields have corresponding read/write calls. Narrow writes check their range before changing output. ReadDouble/WriteDouble preserve bit patterns, including signed zero and NaN payloads.