LZ4

Contents

1. LZ4#

group LZ4_API

LZ4 is a lossless compression algorithm, providing compression speeds greater than 500 MB/s per core, and scalable with multi-core CPU. It features an extremely fast decoder, with speed in multiple GB/s per core and reaching RAM speed limits on multi-core systems.

This library provides in-memory compression and decompression functions. It gives full buffer control to the user. Compression can be done in:

  • a single step (described in Simple Functions)

  • a single step, reusing a context (described in Advanced Functions)

  • unbounded multiple steps (described in Streaming compression)

APIs in lz4.h generate LZ4-compressed blocks in the format lz4_Block_format. Decompressing such a compressed block requires additional metadata. Exact metadata depends on exact decompression function. In the case of LZ4_decompress_safe(), the metadata includes block’s compressed size, and maximum bound of decompressed size. Each application is free to encode and pass such metadata in whichever way it wants.

API in lz4.h only handle blocks, it cannot generate Frames.

Blocks are different from Frames lz4_Frame_format. The Frames bundle both blocks and metadata in the specified manner. Embedding metadata is required for the compressed data to be self-contained and portable. The Frame format is delivered through the companion API declared in lz4frame.h. The lz4 CLI can only manage Frames.

Helper Functions

int LZ4_versionNumber(void)#

Library Version number. Useful to check Dynamic-link Library (DLL) version. Requires v1.3.0+.

Returns:

Library Version in integer format.

const char *LZ4_versionString(void)#

Library version string. Useful to check DLL version. Requires v1.7.5+.

Returns:

Version in const char* format.

Simple Functions

int LZ4_compress_default(const char *src, char *dst, int srcSize, int dstCapacity)#

Compresses ‘srcSize’ bytes from buffer ‘src’ into allocated ‘dst’ buffer of size ‘dstCapacity’.

Parameters

Direction

Description

src

in

Source buffer, the data you want to compress is copied/or pointed here.

dst

out

Destination buffer, compressed data is kept here, memory should be allocated already.

srcSize

in

Maximum supported value is LZ4_MAX_INPUT_SIZE.

dstCapacity

in

Size of pre-allocated ‘dst’ buffer.

Note

Compression is guaranteed to succeed if ‘dstCapacity’ >= LZ4_compressBound(srcSize). It is recommended to use this setting as the compression runs faster on this setting.

Note

This function is protected against buffer overflow scenarios (never writes outside ‘dst’ buffer nor reads outside ‘source’ buffer).

Warning

If the function cannot compress ‘src’ into a more limited ‘dst’ budget, compression stops immediately and the function result is zero. In which case, ‘dst’ content is undefined (invalid).

Returns:

Result

Description

Success

Returns a positive number (<= dstCapacity) indicating the number of bytes written into the buffer dst.

Fail

Returns 0.

int LZ4_decompress_safe(const char *src, char *dst, int compressedSize, int dstCapacity)#

Decompresses the compressed data pointed by src into dst and returns the number of bytes decompressed into the destination buffer.

Parameters

Direction

Description

src

in

This buffer contains compressed data.

dst

out

This is the destination buffer, the data is decompressed to this buffer.

compressedSize

in

It is the exact complete size of the compressed block.

dstCapacity

in

It is the size of the destination buffer (which must be pre-allocated), presumed to be an upper bound of decompressed size.

Note

1 : This function is protected against malicious data packets : it will never writes outside ‘dst’ buffer, nor read outside ‘source’ buffer, even if the compressed block is maliciously modified to order the decoder to do these actions. In such case, the decoder stops immediately, and considers the compressed block malformed.

Note

2 : compressedSize and dstCapacity must be provided to the function, the compressed block does not contain them. The implementation is free to send / store / derive this information in whichever way is most beneficial. If there is a need for a different format which bundles together both compressed data and its metadata, consider looking at lz4frame.h instead.

Returns:

Result

Description

Success

The number of bytes decompressed into destination buffer (<= dstCapacity)

Fail

If destination buffer is not large enough, decoding will stop and output an error code (negative value).

If the source stream is detected malformed, the function will stop decoding and return a negative result.

Advanced Functions

int LZ4_compressBound(int inputSize)#

Provides the maximum size that LZ4 compression may output in a “worst case” scenario (input data not compressible).

This function is primarily useful for memory allocation purposes (destination buffer size). Macro LZ4_COMPRESSBOUND(isize) is also provided for compilation-time evaluation (stack memory allocation for example).

Parameters

Direction

Description

inputSize

in

Maximum supported value is LZ4_MAX_INPUT_SIZE .

Note

LZ4_compress_default() compresses faster when dstCapacity is >= LZ4_compressBound(srcSize).

Returns:

Result

Description

Success

Returns Maximum output size in a “worst case” scenario

Fail

Returns 0, if input size is incorrect (too large or negative).

int LZ4_compress_fast(const char *src, char *dst, int srcSize, int dstCapacity, int acceleration)#

Same as LZ4_compress_default(), but allows the selection of “acceleration” factor.

Parameters

Direction

Description

src

in

Source buffer, the data you want to compress is copied/or pointed here.

dst

out

Destination buffer, compressed data is kept here, memory should be allocated already.

srcSize

in

Maximum supported value is LZ4_MAX_INPUT_SIZE.

dstCapacity

in

Size of buffer ‘dst’ (which must be already allocated).

acceleration

in

Values <= 0 will be replaced by LZ4_ACCELERATION_DEFAULT (currently == 1, see lz4.c).

Values > LZ4_ACCELERATION_MAX will be replaced by LZ4_ACCELERATION_MAX (currently == 65537, see lz4.c).

The larger the acceleration value, the faster the algorithm, but also the lesser the compression. It’s a trade-off.

It can be fine-tuned with each successive value providing roughly +~3% to speed.

An acceleration value of “1” is the same as regular LZ4_compress_default().

Returns:

Result

Description

Success

Returns a positive number (<= dstCapacity) indicating the number of bytes written into the buffer dst.

Fail

Returns 0.

int LZ4_sizeofState(void)#

Get the memory to be allocated for its state.

Returns:

The amount of memory which must be allocated for its state.

int LZ4_compress_fast_extState(void *state, const char *src, char *dst, int srcSize, int dstCapacity, int acceleration)#

Same as LZ4_compress_fast(), using an externally allocated memory space for its state.

Use LZ4_sizeofState() to know how much memory must be allocated, and allocate it on 8-bytes boundaries (using malloc() typically). Then, provide this buffer as void* state to compression function.

Parameters

Direction

Description

state

in,out

It acts as a handle.

src

in

Source buffer, the data which you want to compress is copied/or pointed here.

dst

out

Destination buffer, compressed data is kept here, memory should be allocated already.

srcSize

in

Maximum supported value is LZ4_MAX_INPUT_SIZE.

dstCapacity

in

Size of buffer ‘dst’ (which must be already allocated).

acceleration

in

The larger the acceleration value, the faster the algorithm, but also the lesser the compression.

It’s a trade-off. It can be fine-tuned, with each successive value providing roughly +~3% to speed.

An acceleration value of “1” is the same as regular LZ4_compress_default().

Values <= 0 will be replaced by LZ4_ACCELERATION_DEFAULT (currently == 1, see lz4.c).

Values > LZ4_ACCELERATION_MAX will be replaced by LZ4_ACCELERATION_MAX (currently == 65537, see lz4.c).

Returns:

Result

Description

Success

Returns a positive number (<= dstCapacity) indicating the number of bytes written into the buffer dst.

Fail

Returns 0.

int LZ4_compress_destSize(const char *src, char *dst, int *srcSizePtr, int targetDstSize)#

This function either compresses the entire ‘src’ content into ‘dst’ if it’s large enough or fills ‘dst’ buffer completely with as much data as possible from ‘src’. Reverse the logic : Compresses as much data as possible from the ‘src’ buffer into the allocated buffer ‘dst’, of size >= ‘targetDestSize’.

Note: Acceleration parameter is fixed to “default”.

Parameters

Direction

Description

src

in

Source buffer, the data you want to compress is copied/or pointed here.

dst

out

Destination buffer, compressed data is kept here, memory should be pre-allocated.

srcSizePtr

in,out

Will be modified to indicate how many bytes were read from ‘src’ to fill ‘dst’. New value is necessarily <= input value.

Size

in

Size of buffer ‘dst’ (which must be already allocated).

Warning

From v1.8.2 to v1.9.1, this function had a bug (fixed un v1.9.2+): Sometimes, the produced compressed content must be decompressed into a destination buffer larger by at least 1 byte than the content to decompress. If an application uses LZ4_compress_destSize(), it’s highly recommended to update liblz4 to v1.9.2 or better. If this can’t be done or ensured, the receiving decompression function should provide a dstCapacity > decompressedSize, by at least 1 byte. For more information, refer to lz4/lz4#859.

Returns:

Result

Description

Success

Returns Nb bytes written into dst (<= targetDestSize).

Fail

Returns 0 on fail .

int LZ4_decompress_safe_partial(const char *src, char *dst, int srcSize, int targetOutputSize, int dstCapacity)#

Decompress an LZ4 compressed block, of size ‘srcSize’ at the position ‘src’ into destination buffer ‘dst’ of size ‘dstCapacity’. Up to ‘targetOutputSize’ bytes will be decoded.

The function stops decoding on reaching this objective. This can be useful to boost performance whenever only the beginning of a block is required.

Note

1 : return can be < targetOutputSize, if compressed block contains less data.

Note

2 : targetOutputSize must be <= dstCapacity.

Note

3 : this function effectively stops decoding on reaching targetOutputSize, so dstCapacity is kind of redundant. This is because in older versions of this function, decoding operation would still write complete sequences. Therefore, there was no guarantee that it would stop writing at exactly targetOutputSize, it could write more bytes, though only up to dstCapacity. Some “margin” used to be required for this operation to work properly. Thankfully, this is no longer necessary. The function nonetheless keeps the same signature, in an effort to preserve API compatibility.

Note

4 : If srcSize is the exact size of the block, then targetOutputSize can be any value, including larger than the block’s decompressed size. The function will, at most, generate block’s decompressed size.

Note

5 : If srcSize is larger than block’s compressed size, then targetOutputSize MUST be <= block’s decompressed size. Otherwise, silent corruption will occur.

Returns:

Result

Description

Success

The number of bytes decoded in dst (<= targetOutputSize)

Fail

If source stream is detected malformed, function returns a negative result.

Streaming Compression Functions

LZ4_stream_t *LZ4_createStream(void)#

Creates LZ4_stream_t, allocates memory dynamically and returns its memory address.

Parameters:

void –

Returns:

A pointer of LZ4_stream_t type whose memmory has been allocated dynamically.

int LZ4_freeStream(LZ4_stream_t *streamPtr)#

Frees the memory pointed by streamPtr .

Parameters

Direction

Description

StreamPtr

in,out

LZ4_stream_t* type streaming compression tracking context. A tracking context can be re-used multiple times. Declare or create LZ4_stream_t using LZ4_createStream(), which is recommended.

Returns:

int 0 .

void LZ4_resetStream_fast(LZ4_stream_t *streamPtr)#

Use this to prepare an LZ4_stream_t for a new chain of dependent blocks (for example, LZ4_compress_fast_continue()).

An LZ4_stream_t must be initialized once before usage. This is automatically done when created by LZ4_createStream(). However, should the LZ4_stream_t be simply declared on stack (for example), it’s necessary to initialize it first, using LZ4_initStream().

After init, start any new stream with LZ4_resetStream_fast(). A same LZ4_stream_t can be re-used multiple times consecutively and compress multiple streams, provided that it starts each new stream with LZ4_resetStream_fast().

LZ4_resetStream_fast() is much faster than LZ4_initStream(), but is not compatible with memory regions containing garbage data.

Parameters

Direction

Description

StreamPtr

in,out

LZ4_stream_t* type streaming compression tracking context. A tracking context can be re-used multiple times. Declare or create LZ4_stream_t using LZ4_createStream(), which is recommended.

Attention

This function is for v1.9.0+.

Note

It’s only useful to call LZ4_resetStream_fast() in the context of streaming compression. The extState functions perform their own resets. Invoking LZ4_resetStream_fast() before is redundant, and even counterproductive.

Returns:

void .

int LZ4_loadDict(LZ4_stream_t *streamPtr, const char *dictionary, int dictSize)#

Use this function to reference a static dictionary into LZ4_stream_t.

The dictionary must remain available during compression. LZ4_loadDict() triggers a reset, so any previous data will be forgotten. The same dictionary will have to be loaded on decompression side for successful decoding. Dictionary are useful for better compression of small data (KB range). While LZ4 accept any input as dictionary, results are generally better when using Zstandard’s Dictionary Builder. Loading a size of 0 is allowed, and is the same as reset.

Parameters

Direction

Description

streamPtr

in,out

LZ4_stream_t* type streaming type compression tracking context. A tracking context can be re-used multiple times. Declare or create LZ4_stream_t using LZ4_createStream(), which is recommended. Initialize LZ4_stream_t using LZ4_initStream().

dictionary

in,out

Dictionary buffer

dictSize

in

Size of dictionary

Returns:

Result

Description

Success

Loaded dictionary size, in bytes (<= 64 KB).

Fail

-1 if (streamPtr == NULL) or (dictionary == NULL && dictSize >= sizeof(reg_t)).

int LZ4_compress_fast_continue(LZ4_stream_t *streamPtr, const char *src, char *dst, int srcSize, int dstCapacity, int acceleration)#

Compress src content using data from the previously compressed blocks for a better compression ratio. The dst buffer must be already allocated. If dstCapacity >= LZ4_compressBound(srcSize), the compression would succeed and run faster.

Parameters

Direction

Description

state

in,out

LZ4_stream_t* type streaming type compression tracking context. A tracking context can be re-used multiple times. Declare or create LZ4_stream_t using LZ4_createStream(), which is recommended. Initialize LZ4_stream_t using LZ4_initStream().

src

in

Source buffer, the data you want to compress is copied/or pointed here.

dst

out

Destination buffer, compressed data is kept here, memory should be allocated already.

srcSize

in

Maximum supported value is LZ4_MAX_INPUT_SIZE.

dstCapacity

in

Size of buffer ‘dst’ (which must be already allocated).

acceleration

in

The larger the acceleration value, the faster the algorithm, but also the lesser the compression.

It’s a trade-off. It can be fine-tuned, with each successive value providing roughly +~3% to speed.

An acceleration value of “1” is the same as regular LZ4_compress_default().

Values <= 0 will be replaced by LZ4_ACCELERATION_DEFAULT (currently == 1, see lz4.c).

Values > LZ4_ACCELERATION_MAX will be replaced by LZ4_ACCELERATION_MAX (currently == 65537, see lz4.c).

Note

1 : Each invocation to LZ4_compress_fast_continue() generates a new block. Each block has precise boundaries and must be decompressed separately, calling LZ4_decompress_*() with relevant metadata. It’s not possible to append blocks together and expect a single invocation of LZ4_decompress_*() to decompress them together.

Note

2 : The previous 64KB of source data is assumed to remain present, unmodified, at same address in memory !

Note

3 : When input is structured as a double buffer, each buffer can have any size, including < 64 KB. Make sure that buffers are separated, by at least one byte. This construction ensures that each block only depends on previous block.

Note

4 : If input buffer is a ring-buffer, it can have any size, including < 64 KB.

Note

5 : After an error, the stream status is undefined (invalid), it can only be reset or freed.

Returns:

Result

Description

Success

Returns the size of compressed block

Fail

Returns 0 if there is an error (typically, cannot fit into ‘dst’).

int LZ4_saveDict(LZ4_stream_t *streamPtr, char *safeBuffer, int maxDictSize)#

If the last 64KB data would not be available at its current memory location, save it in a safer location (char* safeBuffer).

Parameters

Direction

Description

streamPtr

in,out

LZ4_stream_t* type streaming type compression tracking context. A tracking context can be re-used multiple times. Declare or create LZ4_stream_t using LZ4_createStream(), which is recommended. Initialize LZ4_stream_t using LZ4_initStream().

safeBuffer

in

Buffer where you want to store dictionary.

maxDictSize

in

Size of safeBuffer, memory should be allocated such that dictionary would fit inside safeBuffer.This is schematically equivalent to a memcpy() followed by LZ4_loadDict(),but is much faster, because LZ4_saveDict() doesn’t need to rebuild tables.

LZ4_saveDict() : If previously compressed data block is not guaranteed to remain available at its memory location, save it into a safer place (char* safeBuffer). Note : no need to call LZ4_loadDict() afterwards, dictionary is immediatley usable, one can therefore call LZ4_compress_fast_continue() right after.

Returns:

Result

Description

Success

Saved dictionary size in bytes which is a positive integer (<= maxDictSize)

Fail

Returns 0 if error.

Returns:

: saved dictionary size in bytes (necessarily <= dictSize), or 0 if error.

LZ4_stream_t *LZ4_initStream(void *buffer, size_t size)#

Use LZ4_initStream() to properly initialize a newly declared LZ4_stream_t. It can also initialize any arbitrary buffer of sufficient size, and will return a pointer of the proper type upon initialization.

An LZ4_stream_t structure must be initialized at least once. This is automatically done when invoking LZ4_createStream(), but it’s not when the structure is simply declared on stack (for example).

Note

1: Initialization fails if size and alignment conditions are not respected. In which case, the function will return NULL.

Note

2: An LZ4_stream_t structure guarantees correct alignment and size.

Note

3: Before v1.9.0, use LZ4_resetStream() instead

Warning

Works for v1.9.0+

Returns:

Result

Description

Success

A pointer of proper type upon initialization.

Fail

Initialization fails if size and alignment conditions are not respected. In which case, the function will return NULL.

Streaming Decompression Functions

LZ4_streamDecode_t *LZ4_createStreamDecode(void)#

Creation of streaming decompression tracking context.

LZ4_createStreamDecode() and LZ4_freeStreamDecode() : creation / destruction of streaming decompression tracking context.

A tracking context can be re-used multiple times.

Returns:

A pointer of LZ4_stream_t type whose memmory has been allocated dynamically.

int LZ4_freeStreamDecode(LZ4_streamDecode_t *LZ4_stream)#

Frees up the memory occupied by LZ4_stream .

Parameters

Direction

Description

LZ4_stream

in,out

LZ4_streamDecode_t* works as a streaming decompression tracking context. A tracking context can be re-used multiple times.

Returns:

int 0.

int LZ4_setStreamDecode(LZ4_streamDecode_t *LZ4_streamDecode, const char *dictionary, int dictSize)#

Use this function to start decompression of a new stream of blocks.

Parameters

Direction

Description

LZ4_streamDecode

in,out

An LZ4_streamDecode_t context can be allocated once and re-used multiple times.

dictionary

in,out

A dictionary can optionally be set. Use NULL or size 0 for a reset order.

dictSize

in

Size of dictionary

Note

Dictionary is presumed stable : it must remain accessible and unmodified during next decompression.

Returns:

Result

Description

Success

Returns 1 if OK .

Fail

Returns 0 if error.

int LZ4_decoderRingBufferSize(int maxBlockSize)#

In a ring buffer scenario (optional), blocks are presumed decompressed next to each other until there isn’t enough space for the next block (remainingSize < maxBlockSize). After that it resumes from the beginning of ring buffer.

When setting such a ring buffer for streaming decompression, this function provides the minimum size of this ring buffer to be compatible with any source respecting maxBlockSize condition.

Parameters

Direction

Description

maxBlockSize

in,out

The maximum block size of compressed data.

Warning

This works for v1.8.2+ .

Returns:

Result

Description

Success

Minimum ring buffer size.

Fail

Returns 0 if there is an error (invalid maxBlockSize).

int LZ4_decompress_safe_continue(LZ4_streamDecode_t *LZ4_streamDecode, const char *src, char *dst, int srcSize, int dstCapacity)#

These decoding functions allow decompression of consecutive blocks in “streaming” mode.

A block is an unsplittable entity, it must be presented entirely to a decompression function. Decompression functions only accepts one block at a time. The last 64KB of previously decoded data must remain available and unmodified at the memory position where they were decoded. If less than 64KB of data has been decoded, all the data must be present.

Special : if decompression side sets a ring buffer, it must respect one of the following conditions :

  • Decompression buffer size is at least LZ4_decoderRingBufferSize(maxBlockSize). maxBlockSize is the maximum size of any single block. It can have any value > 16 bytes. In which case, encoding and decoding buffers do not need to be synchronized. Actually, data can be produced by any source compliant with LZ4 format specification, and respecting maxBlockSize.

  • Synchronized mode : Decompression buffer size is exactly the same as compression buffer size, and follows exactly same update rule (block boundaries at same positions), and decoding function is provided with exact decompressed size of each block (exception for last block of the stream), then decoding & encoding ring buffer can have any size, including small ones ( < 64 KB).

  • Decompression buffer is larger than encoding buffer, by a minimum of maxBlockSize more bytes. In which case, encoding and decoding buffers do not need to be synchronized, and encoding ring buffer can have any size, including small ones ( < 64 KB).

Parameters

Direction

Description

LZ4_streamDecode

in,out

An LZ4_streamDecode_t context can be allocated once and re-used multiple times.

src

in

This buffer contains compressed data.

dst

out

Destination buffer, the data is decompressed to this buffer.

srcSize

in

It is the exact complete size of the src.

dstCapacity

in

It is the size of destination buffer (which must be already allocated), presumed an upper bound of decompressed size.

Note

Whenever these conditions are not possible, save the last 64KB of decoded data into a safe buffer where it can’t be modified during decompression, then indicate where this data is saved using LZ4_setStreamDecode(), before decompressing next block.

Returns:

Result

Description

Success

Returns the number of bytes decoded in dst which is positive (<= targetOutputSize)

Fail

If source stream is detected malformed, function returns a negative result.

int LZ4_decompress_safe_usingDict(const char *src, char *dst, int srcSize, int dstCapacity, const char *dictStart, int dictSize)#

These decoding functions work the same as a combination of LZ4_setStreamDecode() followed by LZ4_decompress_*_continue()

They are stand-alone, and don’t need an LZ4_streamDecode_t structure. Dictionary is presumed stable : it must remain accessible and unmodified during decompression. Performance tip : Decompression speed can be substantially increased when dst == dictStart + dictSize.

Parameters

Direction

Description

src

in

This buffer contains compressed data.

dst

out

Destination buffer, the data is decompressed to this buffer.

srcSize

in

It is the exact complete size of the src.

dstCapacity

in

It is the size of destination buffer (which must be already allocated), presumed an upper bound of decompressed size.

dictStart

in,out

A dictionary can optionally be set. Use NULL or size 0 for a reset order.

dictSize

in

Size of dictionary.

Returns:

Result

Description

Success

The number of bytes decoded in dst (<= targetOutputSize)

Fail

If source stream is detected malformed, function returns a negative result.

int LZ4_decompress_safe_partial_usingDict(const char *src, char *dst, int compressedSize, int targetOutputSize, int maxOutputSize, const char *dictStart, int dictSize)#

Defines

LZ4_COMPRESSBOUND(isize)#

Macro LZ4_COMPRESSBOUND(isize) is provided for compilation-time evaluation (stack memory allocation) alternative of LZ4_compressBound()

Typedefs

typedef union LZ4_stream_u LZ4_stream_t#

A tracking context can be re-used multiple times. Declare or create LZ4_stream_t using LZ4_createStream(), which is recommended.

typedef union LZ4_streamDecode_u LZ4_streamDecode_t#

It is used to track the LZ4 stream during decompression.

1.1. LZ4 Frame Format APIs#

group LZ4FRAME_API

LZ4F is a stand-alone API able to create and decode LZ4 frames conformant with specification v1.6.1 in lz4/lz4 . Generated frames are compatible with lz4 CLI, and designed to be interoperable with any system.

lz4frame.h implements LZ4 frame specification: see doc/lz4_Frame_format.md . * LZ4F also offers streaming capabilities.

lz4.h is not required when using lz4frame.h, except to extract common constants such as LZ4_VERSION_NUMBER.

Defines

LZ4F_VERSION#

This number can be used to check for an incompatible API breaking change.

Typedefs

typedef struct LZ4F_cctx_s LZ4F_cctx#
typedef LZ4F_cctx *LZ4F_compressionContext_t#
typedef struct LZ4F_dctx_s LZ4F_dctx#
typedef LZ4F_dctx *LZ4F_decompressionContext_t#
typedef size_t LZ4F_errorCode_t#

Enums

enum LZ4F_blockChecksum_t#

Values:

enumerator LZ4F_noBlockChecksum#
enumerator LZ4F_blockChecksumEnabled#
enum LZ4F_blockMode_t#

Values:

enumerator LZ4F_blockLinked#
enumerator LZ4F_blockIndependent#
enum LZ4F_blockSizeID_t#

The larger the block size, the (slightly) better the compression ratio, though there are diminishing returns. Larger blocks also increase memory usage on both compression and decompression sides.

Values:

enumerator LZ4F_default#
enumerator LZ4F_max64KB#
enumerator LZ4F_max256KB#
enumerator LZ4F_max1MB#
enumerator LZ4F_max4MB#
enum LZ4F_contentChecksum_t#

Values:

enumerator LZ4F_noContentChecksum#
enumerator LZ4F_contentChecksumEnabled#
enum LZ4F_frameType_t#

Values:

enumerator LZ4F_frame#
enumerator LZ4F_skippableFrame#

Functions

size_t LZ4F_compressBegin(LZ4F_cctx *cctx, void *dstBuffer, size_t dstCapacity, const LZ4F_preferences_t *prefsPtr)#

will write the frame header into dstBuffer.

Parameters

Direction

Description

cctx

in

A compression context that can be employed multiple times for consecutive streaming operations.

dstBuffer

out

Destination buffer, compressed data is kept here, memory should be allocated already.

dstCapacity

in

Size of pre-allocated ‘dst’ buffer.

prefsPtr

in

An optional pointer which makes it possible to supply advanced compression instructions to streaming interface. It can be replaced by NULL, in which case, the function will assume default preferences.

LZ4F_compressBegin() : init streaming compression AND writes frame header into @dstBuffer. @dstCapacity must be >= LZ4F_HEADER_SIZE_MAX bytes. @preferencesPtr can be NULL, in which case default parameters are selected.

Note

dstCapacity must be >= LZ4F_HEADER_SIZE_MAX bytes.

Returns:

Result

Description

Success

Number of bytes written into dstBuffer.

Failure

An error code if it fails (can be tested using LZ4F_isError()).

Returns:

: number of bytes written into dstBuffer for the header or an error code (can be tested using LZ4F_isError())

size_t LZ4F_compressBound(size_t srcSize, const LZ4F_preferences_t *prefsPtr)#

Provides minimum dstCapacity required to guarantee success of LZ4F_compressUpdate(), given a srcSize and preferences, for a worst case scenario.

When srcSize==0, LZ4F_compressBound() provides an upper bound for LZ4F_flush() and LZ4F_compressEnd() instead.

Parameters

Direction

Description

srcSize

in

Input buffer size.

preferencesPtr

in

An optional pointer which makes it possible to supply advanced compression instructions to streaming interface. It can be replaced by NULL, in which case, the function will assume default preferences.

Note

The result is only valid for a single invocation of LZ4F_compressUpdate(). When invoking LZ4F_compressUpdate() multiple times, if the output buffer is gradually filled up instead of emptied and re-used from its start, one must check if there is enough remaining capacity before each invocation, using LZ4F_compressBound().

Returns:

is always the same for a srcSize and prefsPtr. prefsPtr is optional : when NULL is provided, preferences will be set to cover worst case scenario. tech details :

Returns:

if automatic flushing is not enabled, includes the possibility that internal buffer might already be filled by up to (blockSize-1) bytes. It also includes frame footer (ending + checksum), since it might be generated by LZ4F_compressEnd().

Returns:

doesn’t include frame header, as it was already generated by LZ4F_compressBegin().

size_t LZ4F_compressEnd(LZ4F_cctx *cctx, void *dstBuffer, size_t dstCapacity, const LZ4F_compressOptions_t *cOptPtr)#

To properly finish an LZ4 frame, invoke LZ4F_compressEnd().

It will flush whatever data remained within cctx (like LZ4_flush()) and properly finalize the frame, with an endMark and a checksum.

Parameters

Direction

Description

cctx

in

A compression context that can be employed multiple times for consecutive streaming operations.

dstBuffer

out

Destination buffer, compressed data is kept here, memory should be allocated already.

dstCapacity

in

Size of pre-allocated ‘dst’ buffer. Must be large enough to work properly, value should be >= LZ4F_compressBound(0, prefsPtr)

cOptPtr

in

is optional : NULL can be provided, in which case all options are set to default.

LZ4F_compressEnd() : When you want to properly finish the compressed frame, just call LZ4F_compressEnd(). It will flush whatever data remained within compressionContext (like LZ4_flush()) but also properly finalize the frame, with an endMark and an (optional) checksum. LZ4F_compressOptions_t structure is optional : you can provide NULL as argument.

Note

LZ4F_compressEnd() is guaranteed to be successful when dstCapacity >= LZ4F_compressBound(0, prefsPtr). A successful call to LZ4F_compressEnd() makes cctx available again for another compression task.

Returns:

Result

Description

Success

nb of bytes written into dstBuffer, necessarily >= 4 (endMark)

Failure

An error code if it fails (can be tested using LZ4F_isError()).

Returns:

: the number of bytes written into dstBuffer (necessarily >= 4 (endMark size)) or an error code if it fails (can be tested using LZ4F_isError()) The context can then be used again to compress a new frame, starting with LZ4F_compressBegin().

size_t LZ4F_compressFrame(void *dstBuffer, size_t dstCapacity, const void *srcBuffer, size_t srcSize, const LZ4F_preferences_t *preferencesPtr)#

Compress an entire srcBuffer into a valid LZ4 frame.

The LZ4F_preferences_t structure is optional : you can provide NULL as argument. All preferences will be set to default.

Parameters

Direction

Description

dstBuffer

out

Destination buffer, compressed data is kept here, memory should be allocated already.

dstCapacity

in

Size of pre-allocated ‘dst’ buffer.

srcBuffer

in

Source buffer, the data you want to compress is copied/or pointed here.

srcSize

in

Size of srcBuffer.

preferencesPtr

in

An optional pointer which makes it possible to supply advanced compression instructions to streaming interface. It can be replaced by NULL, in which case, the function will assume default preferences.

LZ4F_compressFrame() : Compress an entire srcBuffer into a valid LZ4 frame, in a single step. dstBuffer MUST be >= LZ4F_compressFrameBound(srcSize, preferencesPtr). The LZ4F_preferences_t structure is optional : you can provide NULL as argument. All preferences will be set to default.

Note

dstCapacity MUST be >= LZ4F_compressFrameBound(srcSize, preferencesPtr).

Returns:

Result

Description

Success

Number of bytes written into dstBuffer.

Failure

An error code if it fails (can be tested using LZ4F_isError()).

Returns:

: number of bytes written into dstBuffer. or an error code if it fails (can be tested using LZ4F_isError())

size_t LZ4F_compressFrameBound(size_t srcSize, const LZ4F_preferences_t *preferencesPtr)#

This function outputs the maximum possible compressed size with LZ4F_compressFrame() given srcSize and preferences.

Parameters

Direction

Description

srcSize

in

Input buffer size.

preferencesPtr

in

An optional pointer which makes it possible to supply advanced compression instructions to streaming interface. It can be replaced by NULL, in which case, the function will assume default preferences.

Note

this result is only usable with LZ4F_compressFrame(). It may also be relevant to LZ4F_compressUpdate() only if no flush() operation is ever performed.

Returns:

Returns the maximum possible compressed size with LZ4F_compressFrame() given srcSize and preferences.

int LZ4F_compressionLevel_max(void)#

Returns maximum compression level, the higher the compression level, the better the compression ratio.

Returns:

Max compression level is returned.

size_t LZ4F_compressUpdate(LZ4F_cctx *cctx, void *dstBuffer, size_t dstCapacity, const void *srcBuffer, size_t srcSize, const LZ4F_compressOptions_t *cOptPtr)#

LZ4F_compressUpdate() can be called repetitively to compress as much data as necessary.

Parameters

Direction

Description

cctx

in

A compression context that can be employed multiple times for consecutive streaming operations.

dstBuffer

out

Destination buffer, compressed data is kept here, memory should be allocated already.

dstCapacity

in

Size of pre-allocated ‘dst’ buffer. Must be large enough to work properly, value should be >= LZ4F_compressBound()

srcBuffer

in

Source buffer, the data you want to compress is copied/or pointed here.

srcSize

in

Size of srcBuffer.

cOptPtr

in

is optional : NULL can be provided, in which case all options are set to default.

Important

dstCapacity MUST be large enough to ensure operation success even in worst case situations. This value is provided by LZ4F_compressBound(). If this condition is not respected, LZ4F_compress() will fail (result is an errorCode). After an error, the state is left in a UB state, and must be re-initialized or freed. If previously an uncompressed block was written, buffered data is flushed before appending compressed data is continued.

LZ4F_compressUpdate() : LZ4F_compressUpdate() can be called repetitively to compress as much data as necessary. When successful, the function always entirely consumes @srcBuffer. src data is either buffered or compressed into @dstBuffer. If previously an uncompressed block was written, buffered data is flushed before appending compressed data is continued. @dstCapacity MUST be >= LZ4F_compressBound(srcSize, preferencesPtr). @compressOptionsPtr is optional : provide NULL to mean “default”.

Returns:

Result

Description

Success

number of bytes written into dstBuffer (it can be zero, meaning input data was just buffered).

Failure

An error code if it fails (can be tested using LZ4F_isError()).

Returns:

: the number of bytes written into dstBuffer. It can be zero, meaning input data was just buffered. or an error code if it fails (which can be tested using LZ4F_isError()) After an error, the state is left in a UB state, and must be re-initialized.

LZ4F_errorCode_t LZ4F_createCompressionContext(LZ4F_cctx **cctxPtr, unsigned version)#

Used to create compression context that can be employed multiple times for consecutive streaming operations.

The first thing to do before compression is to create a compressionContext object, which will keep track of operation state during streaming compression, this task is achieved by this function.

Parameters

Direction

Description

cctxPtr

in,out

a pointer of pointer to fully allocated LZ4F_cctx object, to write the resulting pointer into.

version

in

provided MUST be LZ4F_VERSION. It is intended to track potential version mismatch, notably when using DLL.

LZ4F_createCompressionContext() : The first thing to do is to create a compressionContext object, which will be used in all compression operations. This is achieved using LZ4F_createCompressionContext(), which takes as argument a version and an LZ4F_preferences_t structure. The version provided MUST be LZ4F_VERSION. It is intended to track potential incompatible differences between different binaries. The function will provide a pointer to an allocated LZ4F_compressionContext_t object. If the result LZ4F_errorCode_t is not OK_NoError, there was an error during context creation. Object can release its memory using LZ4F_freeCompressionContext();

Note

cctxPtr MUST be != NULL.

Returns:

Result

Description

Success

Returns LZ4F_OK_NoError on success.

Failure

Returns LZ4F_ERROR_parameter_null if cctxPtr is NULL.

Returns LZ4F_ERROR_allocation_failed if there is problem with allocation of memory.

LZ4F_errorCode_t LZ4F_createDecompressionContext(LZ4F_dctx **dctxPtr, unsigned version)#

Create an LZ4F_dctx object, to track all decompression operations.

The function fills @dctxPtr with the value of a pointer to an allocated and initialized LZ4F_dctx object. dctx memory can be released using LZ4F_freeDecompressionContext();

Parameters

Direction

Description

dctxPtr

in

Pointer to pointer of a LZ4F_dctx object, to track all decompression operations.

version

in

provided MUST be LZ4F_VERSION. It is intended to track potential version mismatch, notably when using DLL.

LZ4F_createDecompressionContext() : Create a decompressionContext object, which will track all decompression operations. Provides a pointer to a fully allocated and initialized LZ4F_decompressionContext object. Object can later be released using LZ4F_freeDecompressionContext().

Returns:

Result

Description

Success

0 is returned if decompression has been completed fully and correctly.

Failure

An errorCode is returned on failure, which can be tested using LZ4F_isError().

Returns:

: if != 0, there was an error during context creation.

size_t LZ4F_decompress(LZ4F_dctx *dctx, void *dstBuffer, size_t *dstSizePtr, const void *srcBuffer, size_t *srcSizePtr, const LZ4F_decompressOptions_t *dOptPtr)#

Call this function repetitively to regenerate data compressed in srcBuffer.

The function requires a valid dctx state. It will read up to *srcSizePtr bytes from srcBuffer, and decompress data into dstBuffer, of capacity *dstSizePtr.

The nb of bytes consumed from srcBuffer will be written into *srcSizePtr (necessarily <= original value). The nb of bytes decompressed into dstBuffer will be written into *dstSizePtr (necessarily <= original value).

The function does not necessarily read all input bytes, so always check value in *srcSizePtr. Unconsumed source data must be presented again in subsequent invocations.

dstBuffer can freely change between each consecutive function invocation. dstBuffer content will be overwritten.

After decompression a hint of how many srcSize bytes LZ4F_decompress() expects for next call. Schematically, it’s the size of the current (or remaining) compressed block + header of next block. Respecting the hint provides some small speed benefit, because it skips intermediate buffers. This is just a hint though, it’s always possible to provide any srcSize.

When provided with more bytes than necessary to decode a frame, LZ4F_decompress() will stop reading exactly at end of current frame, and return 0.

After a decompression error, the dctx context is not resumable. Use LZ4F_resetDecompressionContext() to return to clean state.

After a frame is fully decoded, dctx can be used again to decompress another frame.

Parameters

Direction

Description

dctx

in,out

LZ4F_dctx object, to track all decompression operations.

dstBuffer

out

Decompressed data is written in dstBuffer, dstBuffer can freely change between each consecutive function invocation & content inside it will be overwritten.

dstSizePtr

in,out

While calling this function “*dstSizePtr” provides the length of dstBuffer, after decompression is done nb of bytes decompressed into dstBuffer will be written into *dstSizePtr (necessarily <= original value).

srcBuffer

in

LZ4 Frame format compatible compressed input stream.

srcSizePtr

in,out

It will read up to *srcSizePtr bytes from srcBuffer and write the nb of bytes consumed from srcBuffer into *srcSizePtr (necessarily <= original value).

dOptPtr

in,out

Advanced decompression options, could be NULL in which case default options are assumed.

LZ4F_decompress() : Call this function repetitively to regenerate compressed data in srcBuffer. The function will attempt to decode up to *srcSizePtr bytes from srcBuffer into dstBuffer of capacity *dstSizePtr.

The number of bytes regenerated into dstBuffer will be provided within *dstSizePtr (necessarily <= original value).

The number of bytes effectively read from srcBuffer will be provided within *srcSizePtr (necessarily <= original value). If number of bytes read is < number of bytes provided, then decompression operation is not complete. Remaining data will have to be presented again in a subsequent invocation.

The function result is an hint of the better srcSize to use for next call to LZ4F_decompress. Schematically, it’s the size of the current (or remaining) compressed block + header of next block. Respecting the hint provides a small boost to performance, since it allows less buffer shuffling. Note that this is just a hint, and it’s always possible to any srcSize value. When a frame is fully decoded,

Returns:

Result

Description

Success

When a frame is fully decoded, 0 will be returned (no more data expected), if not a hint of how many srcSize bytes LZ4F_decompress() expects for next call.

Failure

If decompression failed, an error code is returned, which can be tested using LZ4F_isError().

Returns:

will be 0. If decompression failed,

Returns:

is an error code which can be tested using LZ4F_isError().

size_t LZ4F_flush(LZ4F_cctx *cctx, void *dstBuffer, size_t dstCapacity, const LZ4F_compressOptions_t *cOptPtr)#

When data must be generated and sent immediately, without waiting for a block to be completely filled, it’s possible to call LZ4_flush(). It will immediately compress any data buffered within cctx.

Parameters

Direction

Description

cctx

in

A compression context that can be employed multiple times for consecutive streaming operations.

dstBuffer

out

Destination buffer, compressed data is kept here, memory should be allocated already.

dstCapacity

in

Size of pre-allocated ‘dst’ buffer. Must be large enough to work properly, value should be >= LZ4F_compressBound()

cOptPtr

in

is optional : NULL can be provided, in which case all options are set to default.

LZ4F_flush() : When compressed data must be sent immediately, without waiting for a block to be filled, invoke LZ4_flush(), which will immediately compress any remaining data stored within LZ4F_cctx. The result of the function is the number of bytes written into dstBuffer. It can be zero, this means there was no data left within LZ4F_cctx. The function outputs an error code if it fails (can be tested using LZ4F_isError()) LZ4F_compressOptions_t* is optional. NULL is a valid argument.

Note

LZ4F_flush() is guaranteed to be successful when dstCapacity >= LZ4F_compressBound(0, prefsPtr).

Returns:

Result

Description

Success

nb of bytes written into dstBuffer (can be zero, when there is no data stored within cctx)

Failure

An error code if it fails (can be tested using LZ4F_isError()).

LZ4F_errorCode_t LZ4F_freeCompressionContext(LZ4F_cctx *cctx)#

Once all streaming compression jobs are completed, the state object can be released using this function.

Parameters

Direction

Description

cctx

in

A compression context that can be employed multiple times for consecutive streaming operations.

Returns:

Is always successful even if input is NULL pointer (do nothing), so return value can be ignored.

LZ4F_errorCode_t LZ4F_freeDecompressionContext(LZ4F_dctx *dctx)#

dctx memory can be released using LZ4F_freeDecompressionContext();

Result of LZ4F_freeDecompressionContext() indicates current state of decompressionContext when being released. That is, it should be == 0 if decompression has been completed fully and correctly.

Parameters

Direction

Description

dctx

in

LZ4F_dctx object, to track all decompression operations.

Returns:

Result

Description

Success

0 is returned on success.

Failure

Any value not equal to 0.

const char *LZ4F_getErrorName(LZ4F_errorCode_t code)#

return error code string; for debugging

Returns:

Result

Description

Success

Returns a proper error string.

Failure

Returns a string “Unspecified error code”.

size_t LZ4F_getFrameInfo(LZ4F_dctx *dctx, LZ4F_frameInfo_t *frameInfoPtr, const void *srcBuffer, size_t *srcSizePtr)#

This function extracts frame parameters (max blockSize, dictID, etc.). Its usage is optional: user can also invoke LZ4F_decompress() directly.

Extracted information will fill an existing LZ4F_frameInfo_t structure. This can be useful for allocation and dictionary identification purposes.

LZ4F_getFrameInfo() can work in the following situations :

1) At the beginning of a new frame, before any invocation of LZ4F_decompress(). It will decode header from srcBuffer, consuming the header and starting the decoding process.

Input size must be large enough to contain the full frame header. Frame header size can be known beforehand by LZ4F_headerSize(). Frame header size is variable, but is guaranteed to be >= LZ4F_HEADER_SIZE_MIN bytes, and not more than <= LZ4F_HEADER_SIZE_MAX bytes. Hence, blindly providing LZ4F_HEADER_SIZE_MAX bytes or more will always work. It’s allowed to provide more input data than the header size, LZ4F_getFrameInfo() will only consume the header.

If input size is not large enough, aka if it’s smaller than header size, function will fail and return an error code.

2) After decoding has been started, it’s possible to invoke LZ4F_getFrameInfo() anytime to extract already decoded frame parameters stored within dctx.

Note that, if decoding has barely started, and not yet read enough information to decode the header, LZ4F_getFrameInfo() will fail.

The number of bytes consumed from srcBuffer will be updated in *srcSizePtr (necessarily <= original value). LZ4F_getFrameInfo() only consumes bytes when decoding has not yet started, and when decoding the header has been successful. Decompression must then resume from (srcBuffer + *srcSizePtr).

Parameters

Direction

Description

dctx

in,out

LZ4F_dctx object, to track all decompression operations.

frameInfoPtr

out

This is a pointer to an already allocated LZ4F_frameInfo_t structure where frame parameters are copied into.

srcBuffer

in

LZ4 Frame format compatible compressed input stream.

srcSizePtr

in,out

Pointer to size of src buffer.

LZ4F_getFrameInfo() : This function extracts frame parameters (max blockSize, frame checksum, etc.). Usage is optional. Objective is to provide relevant information for allocation purposes. This function works in 2 situations :

  • At the beginning of a new frame, in which case it will decode this information from srcBuffer, and start the decoding process. Amount of input data provided must be large enough to successfully decode the frame header. A header size is variable, but is guaranteed to be <= LZ4F_HEADER_SIZE_MAX bytes. It’s possible to provide more input data than this minimum.

  • After decoding has been started. In which case, no input is read, frame parameters are extracted from dctx. The number of bytes consumed from srcBuffer will be updated within *srcSizePtr (necessarily <= original value). Decompression must resume from (srcBuffer + *srcSizePtr).

Note

in case of error, dctx is not modified. Decoding operation can resume from beginning safely.

Note

frame parameters are copied into an already allocated LZ4F_frameInfo_t structure.

Returns:

: an hint about how many srcSize bytes LZ4F_decompress() expects for next call, or an error code which can be tested using LZ4F_isError() note 1 : in case of error, dctx is not modified. Decoding operations can resume from where they stopped. note 2 : frame parameters are copied into an already allocated LZ4F_frameInfo_t structure.

Returns:

Result

Description

Success

A hint about how many srcSize bytes LZ4F_decompress() expects for next call

Failure

An error code, which can be tested using LZ4F_isError().

unsigned LZ4F_getVersion(void)#
size_t LZ4F_headerSize(const void *src, size_t srcSize)#

v1.9.0+ Provide the header size of a frame starting at src.

srcSize must be >= LZ4F_MIN_SIZE_TO_KNOW_HEADER_LENGTH, which is enough to decode the header length.

Parameters

Direction

Description

src

in

LZ4 Frame format compatible compressed stream .

srcSize

in

Size of src buffer.

LZ4F_headerSize() :

Note

Frame header size is variable, but is guaranteed to be >= LZ4F_HEADER_SIZE_MIN bytes, and <= LZ4F_HEADER_SIZE_MAX bytes.

Returns:

Result

Description

Success

Size of frame header.

Failure

or an error code, which can be tested using LZ4F_isError().

Returns:

: size of frame header or an error code, which can be tested using LZ4F_isError()

unsigned LZ4F_isError(LZ4F_errorCode_t code)#

tells when a function result is an error code

Returns:

Result

Description

Success

Return 1 if the code represents error code.

Failure

Return 0 if non-error code is passed.

void LZ4F_resetDecompressionContext(LZ4F_dctx *dctx)#

added in v1.8.0 In case of an error, the context is left in “undefined” state. In which case, it’s necessary to reset it, before re-using it.

This method can also be used to abruptly stop any unfinished decompression, and start a new one using same context resources.

Parameters

Direction

Description

dctx

out

LZ4F_dctx object, to track all decompression operations.

Warning

NULL pointer cannot be passed.

Returns:

void

struct LZ4F_compressOptions_t#
struct LZ4F_decompressOptions_t#
#include <lz4frame.h>

Advanced decompression options.

Public Members

unsigned reserved0#

idem.

unsigned reserved1#

must be set to zero for forward compatibility.

unsigned skipChecksums#

disable checksum calculation and verification, even when one is present in frame, to save CPU time. Setting this option to 1 once disables all checksums for the rest of the frame.

unsigned stableDst#

pledges that last 64KB decompressed data will remain available unmodified between invocations. This optimization skips storage operations in tmp buffers.

struct LZ4F_frameInfo_t#
#include <lz4frame.h>

makes it possible to set or read frame parameters.

Structure must be first init to 0, using memset() or LZ4F_INIT_FRAMEINFO, setting all parameters to default. It’s then possible to update selectively some parameters

struct LZ4F_preferences_t#
#include <lz4frame.h>

makes it possible to supply advanced compression instructions to streaming interface.

Structure must be first init to 0, using memset() or LZ4F_INIT_PREFERENCES, setting all parameters to default. All reserved fields must be set to zero.