A single facade over GZip, Deflate, BZip2, and ZIP so you can shrink save files, pack mod bundles, and compress network payloads without wiring a different API for each format.
Summary
The Scylla compression utilities are a single facade over four standard compression formats with consistent byte-array, stream, file, batch, and archive entry points. The facade is ScyllaCompression. Pass it bytes, a stream, a file path, or a folder; get the compressed counterpart back through the same shape. The same API decompresses. Every operation shares the same CompressionSettings, the same progress reporting, the same cancellation, the same error-code contract, and the same crypto-integration bridge to Crypto Utils.
Four algorithms are featured:
- GZip and Deflate are always available because they use the built-in
System.IO.Compressiontypes: GZip is the standard self-describing format with the familiar.gzextension, Deflate is the raw stream behind it (slightly smaller, no header, no auto-detection). - BZip2 and ZIP archives require the
com.unity.sharp-zip-libpackage (1.4.1+), which sets theSCYLLA_HAS_SHARPZIPLIBversion define; without that package the corresponding enum values do not exist on the public surface. ZIP is the only algorithm that supports multi-file archives with optional AES-256 entry encryption.
A game eventually needs to shrink a save file or pack a folder and the Scylla compression utils provide those uses in one familiar interface with sensible defaults, batched parallelism for fan-out workloads, progress callbacks for long operations, cooperative cancellation, and a clean handoff to the crypto subsystem when compression and encryption need to be combined. The “what algorithm should I pick?” question is answered with: GZip unless there is a specific reason otherwise.
Use the compression utilities for:
- Shrinking saved games, exported settings, and JSON config files on disk so they take less space and load faster.
- Packing multiple files into a single ZIP archive (level packs, mod bundles, content distributions).
- Compressing network payloads before sending them; combine with Crypto Utils for an encrypted-and-compressed wire format in one call.
- Building offline asset bundles where output size matters more than processing time (use
MaxCompression). - Batch-compressing many independent buffers in parallel (multiple save slots, multiple network frames) without writing the threading code yourself.
Features
- Single
ScyllaCompressionfacade. One static class covers compress, decompress, archive, and inspect across every algorithm. No per-algorithm class to learn; pick the algorithm throughCompressionSettings.Algorithmand the rest is identical. - Four shipped algorithms. GZip and Deflate are always available through
System.IO.Compression. BZip2 and ZIP archives become available whencom.unity.sharp-zip-libis installed (SCYLLA_HAS_SHARPZIPLIBdefine). Pick GZip unless there is a specific reason otherwise. - Five entry-point shapes. Byte arrays, streams (
Compress(Stream, Stream)), files (CompressFile(path, path)), batches (CompressBatch(items)), and archives (CreateArchive,ExtractArchive,ListArchiveEntries,ExtractEntry). The same algorithm and settings flow through every shape. - Async support with cooperative cancellation. Every long-running operation has an
Asynccounterpart that accepts aCancellationToken. Background compression of large saves, network payloads, or archive creation stays off the main thread. - Progress reporting via
CompressionProgress. Callbacks fire with bytes processed, total bytes, and percentage so your UI can render a loading bar without polling. Available on file, stream, and archive operations. - Batch parallelism.
CompressBatch/DecompressBatchprocess many independent buffers across worker threads in one call. The right shape for multi-slot saves, multi-frame network captures, and fan-out asset packing. CompressionSettingsfor fine-grained control. Algorithm, level, password (ZIP), progress callback, and threading options all live on one settings struct. Defaults are sensible; overrides are explicit.- Four
CompressionLeveltiers.None,Fastest,Normal,Maximum. Each maps to the provider’s native range, so the tier choice is portable across algorithms even when the underlying engines differ. - AES-256 ZIP entry encryption. Setting
CompressionSettings.Passwordon a ZIP archive operation encrypts every entry with AES-256. Encrypted archives are interoperable with standard ZIP tooling that supports AES. - Crypto bridge to
ScyllaCrypto. Combined compress-then-encrypt and decrypt-then-decompress paths route through the crypto subsystem in one call, producing the canonical “encrypted save” wire format without two round-trips through byte buffers. - Runtime algorithm introspection.
IsAlgorithmSupported(algorithm)reports package availability;DetectAlgorithm(bytes)recognises GZip, ZIP, and BZip2 by magic number;GetFileExtension(algorithm)returns the conventional extension. Useful when picking an algorithm at runtime from a config value. - Structured error contract.
CompressionExceptioncarries aCompressionErrorCode(algorithm-not-supported, password-wrong, corrupted-data, …) so error handlers can branch on a stable enum rather than parse a message string.
Algorithms
A compressor squeezes redundancy out of a byte stream by replacing repeated patterns with shorter codes. Different algorithms make different trade-offs between how aggressively they look for patterns, how much memory and CPU they consume, and what format they wrap the compressed bytes in. The four algorithms below cover the practical needs of a game project; the facade picks one based on CompressionSettings.Algorithm (defaults to GZip).
Two terms used in the algorithm descriptions below deserve a quick definition:
- Header is a small prefix the algorithm writes before the compressed bytes. It records the algorithm, the original size, and a checksum so a reader can verify and decompress without out-of-band metadata. GZip and ZIP have headers; raw Deflate does not, which is what makes it auto-detection-unfriendly.
- Compression ratio is the ratio of compressed size to original size, often expressed as a percentage. Lower is better. Text-heavy inputs (JSON, XML, source code) typically reach 10-30%. Already-compressed inputs (PNG, JPEG, MP3, MP4) usually grow by a few percent because there is no redundancy left to exploit.
| Algorithm | Always available? | File extension | Notes |
|---|---|---|---|
| GZip | Yes (built-in) | .gz | Default and recommended. RFC 1952. Wraps raw Deflate output with a header, optional metadata, and a CRC-32 checksum. Self-describing, so ScyllaCompression.DetectAlgorithm(bytes) recognises it without help. Widely supported by external tools (gzip/gunzip, browsers, build tools). |
| Deflate | Yes (built-in) | .deflate | RFC 1951. The raw compressed stream behind GZip, without the header or checksum. Slightly smaller output (saves roughly 20 bytes per blob) and marginally faster, at the cost of no self-description: DetectAlgorithm cannot recognise it. Use only when both endpoints already agree on the algorithm. |
| BZip2 | Requires com.unity.sharp-zip-lib (SCYLLA_HAS_SHARPZIPLIB) | .bz2 | Generally achieves a better compression ratio than GZip on text-heavy data (5-10% smaller is typical) at the cost of substantially slower compression and decompression. Single-stream only, no multi-file archive support. Worth the speed hit for cold archival, rarely worth it for runtime paths. |
| ZIP | Requires com.unity.sharp-zip-lib (SCYLLA_HAS_SHARPZIPLIB) | .zip | The only algorithm that supports multi-file archives via CreateArchive, ExtractArchive, ListArchiveEntries, and ExtractEntry. When CompressionSettings.Password is set to a non-empty string, every entry is encrypted with AES-256. ZIP is the right format for level packs, mod bundles, and any distributable folder. |
The four CompressionLevel tiers (None, Fastest, Normal, Maximum) map to the underlying provider’s native level range. GZip and Deflate are limited to the three tiers exposed by System.IO.Compression (NoCompression, Fastest, Optimal); Normal and Maximum both land on Optimal for those algorithms, so the difference between the two is provider-specific. BZip2 and ZIP honour the full range and produce visibly different outputs for Normal vs Maximum.
Helpers for runtime introspection live on ScyllaCompression itself: IsAlgorithmSupported(algorithm) checks whether the build has the necessary package; DetectAlgorithm(bytes) returns the algorithm a buffer was compressed with (GZip, ZIP, BZip2 by magic number; Deflate is not detectable); GetFileExtension(algorithm) returns the conventional extension (.gz, .deflate, .zip, .bz2).
Recipes
These recipes cover every public surface in the compression subsystem. Each one is a self-contained answer to a single problem, so find the one that matches what you are building and copy the solution. The first recipe shows the simplest byte-array path that later recipes build upon; if you are new to the API, start there.
Shrink a save file with a one-call round-trip
Problem. Your save files are JSON or a serialized binary blob from Serialization Utils, and you want to reduce their on-disk footprint without adding much ceremony to the save and load paths. A typical JSON save compresses to 10-30% of its original size; a 500 KB save file might come back at 50-80 KB after a default GZip pass.
Solution. Pass raw bytes in, get compressed bytes back, pass them back later, get the original bytes back. The simplest call defaults to GZip at CompressionLevel.Normal. The full original is allocated as a single byte array, so this overload is most appropriate for payloads up to a few megabytes; reach for the stream API in “Compress a large replay or world file without hitching” for larger inputs.
/* Plain bytes to compress. UTF-8 strings, serialized JSON, save buffers, and
* binary payloads all work; compression is content-agnostic. */
byte[] saveData = Encoding.UTF8.GetBytes(saveJson);
/* Compress with the default algorithm (GZip) and level (Normal). The output
* is self-describing GZip, so external tools can decompress it too. */
byte[] compressed = ScyllaCompression.Compress(saveData);
/* Write to disk through the Files & IO utilities. */
File.WriteAllBytes(savePath, compressed);
/* On load: decompress back to the original bytes. */
byte[] loaded = File.ReadAllBytes(savePath);
byte[] restored = ScyllaCompression.Decompress(loaded);
/* Safe variant: returns false on corrupted input or wrong algorithm instead
* of throwing. Use this when the input may be from an untrusted source. */
if (ScyllaCompression.TryDecompress(loaded, out var safe))
{
Log.Info($"Decompressed {safe.Length} bytes.", LogCategory.Core);
}
else
{
Log.Warning("Decompression failed: corrupted or wrong algorithm.", LogCategory.Core);
}
Tune compression for speed, size, or offline archival
Problem. The one-call defaults are right for most runtime saves, but sometimes you need a different trade-off: faster compression for an incremental autosave that fires every 30 seconds, or maximum compression for an offline asset bake where output size matters more than bake time.
Solution. Three pre-built CompressionSettings presets cover almost every gameplay scenario. Clone one when you need a one-off variant; build from scratch when a preset does not get you where you need to be. Always call Validate() on custom instances so configuration errors surface next to the configuration rather than deep in the provider.
/* Three always-available presets. These are shared and immutable; never mutate them. */
var defaultSet = CompressionSettings.Default; /* GZip, Normal, 64 KB buffer */
var fastSet = CompressionSettings.Fast; /* GZip, Fastest, 256 KB buffer */
var maxSet = CompressionSettings.MaxCompression; /* GZip, Maximum, 64 KB buffer */
byte[] payload = Encoding.UTF8.GetBytes("Repeated content. Repeated content. Repeated content.");
/* Pick the preset that matches the workload. */
byte[] balanced = ScyllaCompression.Compress(payload, defaultSet);
byte[] quickSave = ScyllaCompression.Compress(payload, fastSet); /* autosave hot path */
byte[] tightPack = ScyllaCompression.Compress(payload, maxSet); /* offline asset bake */
/* Custom settings. Clone a preset rather than mutating it. */
var custom = new CompressionSettings
{
Algorithm = CompressionAlgorithm.GZip,
Level = CompressionLevel.Maximum,
BufferSize = 256 * 1024, /* 256 KB; helps on large sequential reads */
};
/* Validate up front so invalid buffer sizes or parallelism counts raise
* CompressionException(InvalidSettings) here, not somewhere inside the provider. */
custom.Validate();
byte[] customCompressed = ScyllaCompression.Compress(payload, custom);
/* Clone a preset to derive a one-off variant without affecting other callers. */
var fastWithBiggerBuffer = CompressionSettings.Fast.Clone();
fastWithBiggerBuffer.BufferSize = 1 * 1024 * 1024; /* 1 MB for very large sequential writes */
Compress a large replay or world file without hitching
Problem. Your game records a replay or serializes a full open-world chunk, and the resulting blob is tens or hundreds of megabytes. Pulling the whole thing into a single byte[] before compressing would pressure the LOH (large object heap) and produce a visible hitch on mobile. You want to compress it from a stream to a stream so only one buffer is in memory at a time.
Solution. The stream overloads compress from any readable Stream to any writable Stream using the configured BufferSize for chunked copies. An optional IProgress<CompressionProgress> callback fires periodically with bytes processed so you can update a loading bar.
/* Construct a progress callback. Use new Progress<T> on the UI thread so
* the delegate is marshalled back automatically; compression calls from
* worker threads then update the UI safely. */
var progress = new Progress<CompressionProgress>(p =>
{
if (p.PercentComplete >= 0)
{
Log.Info($"{p.BytesProcessed}/{p.TotalBytes} ({p.PercentComplete:F1}%)", LogCategory.Core);
}
else
{
Log.Info($"{p.BytesProcessed} bytes processed", LogCategory.Core);
}
});
/* Stream-to-stream compression. Either side can be any Stream: MemoryStream,
* FileStream, NetworkStream, etc. The destination is left open after the call. */
using var source = File.OpenRead("replay.bin");
using var dest = File.Create("replay.bin.gz");
ScyllaCompression.CompressStream(source, dest, CompressionSettings.Default, progress);
/* Stream decompression has the same shape. */
using var compressedSource = File.OpenRead("replay.bin.gz");
using var decompressedDest = File.Create("replay.bin");
ScyllaCompression.DecompressStream(compressedSource, decompressedDest, CompressionSettings.Default, progress);
/* Overloads without settings use the default algorithm. */
ScyllaCompression.CompressStream(source, dest);
ScyllaCompression.DecompressStream(compressedSource, decompressedDest);
Compress or decompress files on disk
Problem. Both ends of your operation already live on disk: a JSON config you want to compress before shipping, a .gz log file from the server you want to expand before parsing. You want the compression to happen without you opening file handles by hand.
Solution. The file-to-file overloads handle the stream wiring internally and also accept a CancellationToken, so a long compression can be aborted from a UI cancel button without leaving partial output behind.
/* Default file compression. You choose the output extension. */
ScyllaCompression.CompressFile(sourcePath: "save.json", destinationPath: "save.json.gz");
/* With settings, progress, and cancellation. */
var cts = new CancellationTokenSource();
var settings = new CompressionSettings
{
Algorithm = CompressionAlgorithm.GZip,
Level = CompressionLevel.Maximum,
};
ScyllaCompression.CompressFile(
sourcePath: "save.json",
destinationPath: "save.json.gz",
settings: settings,
progress: progress,
cancellationToken: cts.Token);
/* Decompression. The .gz extension is convention; the actual format comes
* from the bytes in the file, not the filename. */
ScyllaCompression.DecompressFile(
sourcePath: "save.json.gz",
destinationPath: "save.json");
/* Helper that maps algorithm to the conventional file extension. */
string ext = ScyllaCompression.GetFileExtension(CompressionAlgorithm.GZip); /* ".gz" */
Keep the main thread free during compression
Problem. Compressing a 500 MB replay on the main thread produces a visible hitch every time. Compression on a large input is CPU-bound, and anything running on the main thread holds up the rest of the frame for the duration. You need the work off the main thread with a clean async/await integration.
Solution. Every sync method has an Async variant that offloads work to a background thread and supports cooperative cancellation. The async overloads return Task (void operations) or Task<byte[]> / Task<byte[][]> (value operations).
public async UniTask SaveCompressedAsync(byte[] saveData, string path, CancellationToken ct)
{
/* Byte-array compression off the main thread. */
byte[] compressed = await ScyllaCompression.CompressAsync(saveData, ct);
await File.WriteAllBytesAsync(path, compressed, ct);
}
public async UniTask<byte[]> LoadCompressedAsync(string path, CancellationToken ct)
{
byte[] compressed = await File.ReadAllBytesAsync(path, ct);
return await ScyllaCompression.DecompressAsync(compressed, ct);
}
/* Stream and file overloads have async counterparts too. */
await ScyllaCompression.CompressStreamAsync(source, dest, ct);
await ScyllaCompression.DecompressStreamAsync(source, dest, ct);
await ScyllaCompression.CompressFileAsync(sourcePath: "save.json", destinationPath: "save.json.gz", ct);
await ScyllaCompression.DecompressFileAsync(sourcePath: "save.json.gz", destinationPath: "save.json", ct);
/* All async overloads accept an optional CompressionSettings and IProgress
* callback the same way the sync variants do. */
await ScyllaCompression.CompressStreamAsync(
source, dest, CompressionSettings.MaxCompression, progress, ct);
Compress multiple save slots in parallel
Problem. Your game supports multiple save slots, and at cloud-sync time you want to compress all of them before uploading. A sequential loop is single-threaded and takes n times the per-slot cost. You want to fan the work out across available CPU cores and collect the results in the same order as the input.
Solution. CompressBatch and DecompressBatch fan the work out across worker threads. Parallelism is bounded by CompressionSettings.MaxDegreeOfParallelism (defaults to Environment.ProcessorCount) and can be disabled entirely by setting UseParallelProcessing = false for deterministic single-threaded debugging.
/* Several independent save payloads to compress before uploading. */
byte[][] saves = new[]
{
Encoding.UTF8.GetBytes(slot1Json),
Encoding.UTF8.GetBytes(slot2Json),
Encoding.UTF8.GetBytes(slot3Json),
};
/* Compress all three in parallel using the default settings. The output
* array has the same length and ordering as the input. */
byte[][] compressed = ScyllaCompression.CompressBatch(saves);
/* Decompress the whole batch. */
byte[][] restored = ScyllaCompression.DecompressBatch(compressed);
/* Custom settings: cap the parallelism or force sequential for debugging. */
var settings = CompressionSettings.Default.Clone();
settings.MaxDegreeOfParallelism = 4; /* cap at 4 threads */
settings.UseParallelProcessing = false; /* force sequential */
byte[][] singleThreaded = ScyllaCompression.CompressBatch(saves, settings);
/* Async variants for non-blocking batch work. */
byte[][] asyncResult = await ScyllaCompression.CompressBatchAsync(saves, ct);
byte[][] asyncRestored = await ScyllaCompression.DecompressBatchAsync(asyncResult, ct);
Pack a mod bundle or level pack into a ZIP
Problem. You have shipped a mod system or a level pack and need to wrap a folder into a single distributable file. ZIP is the format every platform and tool already knows how to handle; you want to pack the folder without wiring ZipArchive yourself and without losing the ability to inspect or partially extract entries.
Solution. CreateArchive walks a source folder and packs every file (respecting FileFilter if set); ExtractArchive reverses the operation; ListArchiveEntries enumerates without extracting; ExtractEntry pulls a single file out. All four methods require the SCYLLA_HAS_SHARPZIPLIB define.
#if SCYLLA_HAS_SHARPZIPLIB
/* Pack an entire folder into a single .zip file. */
ScyllaCompression.CreateArchive(
sourceFolderPath: "Builds/Content",
archivePath: "Builds/Content.zip");
/* With settings: pick the level, filter which files are included, and
* report progress entry-by-entry for a loading bar. */
var settings = CompressionSettings.ZipArchive.Clone();
settings.Level = CompressionLevel.Maximum;
settings.FileFilter = "*.json"; /* only .json files from the folder */
var archiveProgress = new Progress<CompressionProgress>(p =>
{
Log.Info($"[{p.FilesProcessed}/{p.TotalFiles}] {p.CurrentFile}", LogCategory.Core);
});
ScyllaCompression.CreateArchive("Builds/Content", "Builds/Content.zip", settings, archiveProgress);
/* Enumerate without extracting. Useful for previewing a mod bundle
* before the player commits to installing it. */
IReadOnlyList<ArchiveEntryInfo> entries = ScyllaCompression.ListArchiveEntries("Builds/Content.zip");
foreach (var entry in entries)
{
if (entry.IsDirectory)
{
Log.Info($"[DIR] {entry.FullPath}", LogCategory.Core);
}
else
{
Log.Info($"{entry.FullPath}: {entry.CompressedSize}/{entry.UncompressedSize} bytes ({entry.CompressionRatio:F1}%)", LogCategory.Core);
}
}
/* Extract one entry by path. The result is the entry's uncompressed bytes. */
byte[] manifest = ScyllaCompression.ExtractEntry("Builds/Content.zip", "manifest.json");
/* Extract one entry directly to a file path on disk. */
ScyllaCompression.ExtractEntry(
archivePath: "Builds/Content.zip",
entryPath: "manifest.json",
destinationPath: "Temp/manifest.json");
/* Extract the entire archive into a folder. OverwriteExisting defaults to
* false; set it to true when you want to silently replace existing files. */
ScyllaCompression.ExtractArchive(
archivePath: "Builds/Content.zip",
destinationFolderPath: "Temp/Content");
/* Async variants exist for every archive operation. */
await ScyllaCompression.CreateArchiveAsync("Builds/Content", "Builds/Content.zip", settings, archiveProgress, ct);
await ScyllaCompression.ExtractArchiveAsync("Builds/Content.zip", "Temp/Content", settings, archiveProgress, ct);
byte[] entryBytes = await ScyllaCompression.ExtractEntryAsync("Builds/Content.zip", "manifest.json", ct);
#endif
Ship a ZIP with AES-256 entry encryption
Problem. You have a mod that ships proprietary assets, a save bundle bound for a shared server, or a debug payload that includes sensitive profiling data. You want the archive contents to be private, but you still need the ZIP format for tooling compatibility.
Solution. When CompressionSettings.Password is set to a non-empty string and the algorithm is Zip, every entry in the archive is encrypted with AES-256. Reading the archive requires the same password in the settings; the wrong password fails with CompressionErrorCode.WrongPassword. The password is per-archive, not per-entry.
#if SCYLLA_HAS_SHARPZIPLIB
/* Create an encrypted archive. Every entry is AES-256-encrypted with the
* given password. */
var encrypted = CompressionSettings.ZipArchive.Clone();
encrypted.Password = "correct horse battery staple";
ScyllaCompression.CreateArchive("Builds/SecretContent", "Builds/SecretContent.zip", encrypted);
/* Extract requires the same password. */
ScyllaCompression.ExtractArchive("Builds/SecretContent.zip", "Temp/Secret", encrypted);
/* ListArchiveEntries works without the password, but IsEncrypted on each
* entry is true so you can warn the user before attempting extraction. */
var entries = ScyllaCompression.ListArchiveEntries("Builds/SecretContent.zip");
foreach (var entry in entries)
{
if (entry.IsEncrypted)
{
Log.Info($"Encrypted entry: {entry.FullPath}", LogCategory.Core);
}
}
#endif
Encrypt and compress a save payload in one call
Problem. Your save data carries sensitive progress information and you want it both small and private. There is a canonical pairing here: compress first, then encrypt. The order matters because encrypted bytes look like random noise and do not compress, so reversing the steps wastes the compression entirely. Writing the two calls every time gets repetitive and easy to get wrong.
Solution. CompressAndEncrypt and DecryptAndDecompress chain the two operations with sensible defaults. The four-argument overloads accept full CompressionSettings and CryptoSettings for tight control over both passes.
/* One-call combined helper. Compresses with default settings, then
* encrypts with the default Crypto algorithm (AES-GCM on most platforms). */
byte[] secured = ScyllaCompression.CompressAndEncrypt(saveData, password: "pwd");
/* Reverse: decrypt then decompress. */
byte[] restored = ScyllaCompression.DecryptAndDecompress(secured, password: "pwd");
/* Custom settings on both sides for tighter control. */
byte[] tightlySecured = ScyllaCompression.CompressAndEncrypt(
data: saveData,
password: "pwd",
compressionSettings: CompressionSettings.MaxCompression,
cryptoSettings: CryptoSettings.Secure);
byte[] tightlyRestored = ScyllaCompression.DecryptAndDecompress(
data: tightlySecured,
password: "pwd",
compressionSettings: CompressionSettings.MaxCompression,
cryptoSettings: CryptoSettings.Secure);
/* Async variants. */
byte[] asyncSecured = await ScyllaCompression.CompressAndEncryptAsync(saveData, "pwd", ct);
byte[] asyncRestored = await ScyllaCompression.DecryptAndDecompressAsync(asyncSecured, "pwd", ct);
Use the fluent byte-array extensions
Problem. Your code already holds the byte buffer in a local variable and you find the static-method call shape clunky. You want to chain the compression off the buffer so the intent reads more naturally.
Solution. CompressionExtensions mirrors the static facade as extension methods on byte[] and byte[][]. There is no behavioral difference; the extensions exist to read more naturally at call sites that already work with byte arrays.
using Scylla.Core.Util.Compression;
/* Fluent compression on byte arrays. */
byte[] saveData = Encoding.UTF8.GetBytes("payload");
byte[] compressed = saveData.Compress();
byte[] withSettings = saveData.Compress(CompressionSettings.MaxCompression);
byte[] withAlgo = saveData.Compress(CompressionAlgorithm.GZip);
byte[] decompressed = compressed.Decompress();
/* Combined compress+encrypt extensions. */
byte[] secured = saveData.CompressAndEncrypt("pwd");
byte[] restored = secured.DecryptAndDecompress("pwd");
/* Batch extensions. */
byte[][] inputs = new[] { saveData, saveData, saveData };
byte[][] outBatch = inputs.CompressBatch();
byte[][] back = outBatch.DecompressBatch();
/* Async variants. */
byte[] asyncOut = await saveData.CompressAsync(ct);
byte[] asyncBack = await asyncOut.DecompressAsync(ct);
Detect an unknown compressed format at runtime
Problem. Your game downloads asset bundles from a CDN, loads saves from a previous version, or processes third-party exports, and you do not always know which algorithm was used. You need to confirm the format before attempting to decompress, and you want to guard against running BZip2 or ZIP operations on a build that does not have the SharpZipLib package.
Solution. DetectAlgorithm reads the first few bytes (the format’s magic number) and returns the algorithm or null if no match. IsAlgorithmSupported checks whether a particular algorithm is available in the current build. GetFileExtension returns the conventional extension.
/* Identify a compressed blob by its magic number. Returns null for raw
* Deflate (which has no magic number) or for non-compressed input. */
CompressionAlgorithm? detected = ScyllaCompression.DetectAlgorithm(unknownBytes);
if (detected.HasValue)
{
Log.Info($"Detected: {detected.Value}", LogCategory.Core);
}
else
{
Log.Warning("Not a recognised compressed format (or raw Deflate).", LogCategory.Core);
}
/* Confirm an algorithm is supported before requesting it. Avoids an
* UnsupportedAlgorithm exception in builds without the SharpZipLib package. */
if (!ScyllaCompression.IsAlgorithmSupported(CompressionAlgorithm.GZip))
{
Log.Error("GZip should always be available; something is wrong.", LogCategory.Core);
}
/* Conventional extension for each algorithm. */
string ext = ScyllaCompression.GetFileExtension(CompressionAlgorithm.GZip); /* ".gz" */
/* Static configuration. Set once at startup to change the default for the
* whole project. These properties are not synchronized; set them before
* spawning threads that call compression. */
ScyllaCompression.DefaultAlgorithm = CompressionAlgorithm.GZip;
ScyllaCompression.DefaultLevel = CompressionLevel.Normal;
Handle compression errors cleanly
Problem. Anything that touches the filesystem or a network stream can fail: a save file gets corrupted during a bad write, a downloaded asset bundle arrives truncated, or a user builds without SharpZipLib and requests ZIP. You want structured error handling that branches on the failure category rather than parsing message strings.
Solution. Every failure raises CompressionException with a CompressionErrorCode. Inspect the code rather than parsing the message. TryDecompress covers the common case of “the bytes might be wrong”; reserve try/catch for paths where decompression is expected to succeed and a failure represents a programming error or hostile input.
try
{
byte[] restored = ScyllaCompression.Decompress(compressed);
UseRestoredData(restored);
}
catch (CompressionException ex) when (ex.ErrorCode == CompressionErrorCode.CorruptedData)
{
/* Truncated buffer, wrong algorithm, or unrelated bytes. */
Log.Warning("Corrupted compressed data.", LogCategory.Core);
}
catch (CompressionException ex) when (ex.ErrorCode == CompressionErrorCode.UnsupportedAlgorithm)
{
/* Asking for ZIP or BZip2 without the sharp-zip-lib package. */
Log.Error($"Algorithm not available: {ex.Message}", LogCategory.Core);
}
catch (CompressionException ex) when (ex.ErrorCode == CompressionErrorCode.WrongPassword)
{
/* Wrong password on an encrypted ZIP. Prompt the player to retry. */
Log.Warning("Wrong archive password.", LogCategory.Core);
}
catch (CompressionException ex)
{
/* Any other compression failure; ex.ErrorCode pinpoints the cause. */
Log.Error($"Compression error ({ex.ErrorCode}): {ex.Message}", LogCategory.Core);
}
/* CompressionResult and CompressionResult<T> are also public types for
* callers that prefer the result pattern over exceptions. */
CompressionResult<byte[]> wrapped;
try
{
wrapped = CompressionResult<byte[]>.Success(ScyllaCompression.Compress(saveData));
}
catch (CompressionException ex)
{
wrapped = CompressionResult<byte[]>.FromException(ex);
}
if (wrapped.IsSuccess)
{
byte[] compressedSave = wrapped.Value;
}
API Sketch
The compression surface has one facade (ScyllaCompression), one extension class (CompressionExtensions), one settings type (CompressionSettings), three enums (algorithm, level, error code), two progress/result structs (CompressionProgress, ArchiveEntryInfo), and one exception type. Every operation that exists in a synchronous form also exists in an asynchronous form with a CancellationToken parameter. Archive operations exist only when SCYLLA_HAS_SHARPZIPLIB is defined.
public static class ScyllaCompression
{
/* Static configuration applied when no per-call CompressionSettings is supplied.
* Set once at startup before spawning threads. */
public static CompressionAlgorithm DefaultAlgorithm { get; set; } // default GZip
public static CompressionLevel DefaultLevel { get; set; } // default Normal
/* Byte arrays. */
public static byte[] Compress (byte[] data);
public static byte[] Compress (byte[] data, CompressionSettings settings);
public static byte[] Decompress (byte[] compressedData);
public static byte[] Decompress (byte[] compressedData, CompressionSettings settings);
public static bool TryDecompress (byte[] compressedData, out byte[] data);
public static bool TryDecompress (byte[] compressedData, CompressionSettings settings, out byte[] data);
public static Task<byte[]> CompressAsync (byte[] data, CancellationToken ct = default);
public static Task<byte[]> CompressAsync (byte[] data, CompressionSettings settings, CancellationToken ct = default);
public static Task<byte[]> DecompressAsync (byte[] compressedData, CancellationToken ct = default);
public static Task<byte[]> DecompressAsync (byte[] compressedData, CompressionSettings settings, CancellationToken ct = default);
/* Batch (parallel) byte-array compression. */
public static byte[][] CompressBatch (byte[][] dataArrays);
public static byte[][] CompressBatch (byte[][] dataArrays, CompressionSettings settings);
public static byte[][] DecompressBatch (byte[][] compressedArrays);
public static byte[][] DecompressBatch (byte[][] compressedArrays, CompressionSettings settings);
public static Task<byte[][]> CompressBatchAsync (byte[][] dataArrays, CancellationToken ct = default);
public static Task<byte[][]> DecompressBatchAsync(byte[][] compressedArrays, CancellationToken ct = default);
/* Streams. */
public static void CompressStream (Stream source, Stream destination);
public static void CompressStream (Stream source, Stream destination,
CompressionSettings settings,
IProgress<CompressionProgress> progress = null,
CancellationToken ct = default);
public static void DecompressStream(Stream source, Stream destination);
public static void DecompressStream(Stream source, Stream destination,
CompressionSettings settings,
IProgress<CompressionProgress> progress = null,
CancellationToken ct = default);
public static Task CompressStreamAsync (Stream source, Stream destination, CancellationToken ct = default);
public static Task CompressStreamAsync (Stream source, Stream destination, CompressionSettings settings,
IProgress<CompressionProgress> progress = null, CancellationToken ct = default);
public static Task DecompressStreamAsync(Stream source, Stream destination, CancellationToken ct = default);
public static Task DecompressStreamAsync(Stream source, Stream destination, CompressionSettings settings,
IProgress<CompressionProgress> progress = null, CancellationToken ct = default);
/* Files. */
public static void CompressFile (string sourcePath, string destinationPath);
public static void CompressFile (string sourcePath, string destinationPath,
CompressionSettings settings,
IProgress<CompressionProgress> progress = null,
CancellationToken ct = default);
public static void DecompressFile(string sourcePath, string destinationPath);
public static void DecompressFile(string sourcePath, string destinationPath,
CompressionSettings settings,
IProgress<CompressionProgress> progress = null,
CancellationToken ct = default);
public static Task CompressFileAsync (string sourcePath, string destinationPath, CancellationToken ct = default);
public static Task DecompressFileAsync(string sourcePath, string destinationPath, CancellationToken ct = default);
/* Archives (ZIP only; require SCYLLA_HAS_SHARPZIPLIB). */
public static void CreateArchive (string sourceFolderPath, string archivePath);
public static void CreateArchive (string sourceFolderPath, string archivePath,
CompressionSettings settings,
IProgress<CompressionProgress> progress = null,
CancellationToken ct = default);
public static void ExtractArchive(string archivePath, string destinationFolderPath);
public static void ExtractArchive(string archivePath, string destinationFolderPath,
CompressionSettings settings,
IProgress<CompressionProgress> progress = null,
CancellationToken ct = default);
public static IReadOnlyList<ArchiveEntryInfo> ListArchiveEntries(string archivePath);
public static byte[] ExtractEntry(string archivePath, string entryPath);
public static void ExtractEntry(string archivePath, string entryPath, string destinationPath);
public static void ExtractEntry(string archivePath, string entryPath, string destinationPath, CompressionSettings settings);
public static Task CreateArchiveAsync (string sourceFolderPath, string archivePath, CancellationToken ct = default);
public static Task ExtractArchiveAsync (string archivePath, string destinationFolderPath, CancellationToken ct = default);
public static Task<byte[]> ExtractEntryAsync(string archivePath, string entryPath, CancellationToken ct = default);
public static Task ExtractEntryAsync(string archivePath, string entryPath, string destinationPath, CancellationToken ct = default);
/* Crypto integration (combined compress + encrypt). */
public static byte[] CompressAndEncrypt (byte[] data, string password);
public static byte[] CompressAndEncrypt (byte[] data, string password,
CompressionSettings compressionSettings,
CryptoSettings cryptoSettings);
public static byte[] DecryptAndDecompress (byte[] data, string password);
public static byte[] DecryptAndDecompress (byte[] data, string password,
CompressionSettings compressionSettings,
CryptoSettings cryptoSettings);
public static Task<byte[]> CompressAndEncryptAsync (byte[] data, string password, CancellationToken ct = default);
public static Task<byte[]> DecryptAndDecompressAsync(byte[] data, string password, CancellationToken ct = default);
/* Runtime introspection. */
public static bool IsAlgorithmSupported(CompressionAlgorithm algorithm);
public static CompressionAlgorithm? DetectAlgorithm (byte[] compressedData);
public static string GetFileExtension (CompressionAlgorithm algorithm);
}
public static class CompressionExtensions
{
/* Fluent extensions on byte[] and byte[][] mirroring every Compress/Decompress
* variant above (sync, async, settings, algorithm, batch, combined crypto).
* Pure pass-through to ScyllaCompression; behavior is identical. */
}
public sealed class CompressionSettings
{
public CompressionAlgorithm Algorithm { get; set; } // default GZip
public CompressionLevel Level { get; set; } // default Normal
public int BufferSize { get; set; } // default 64 KB
public bool IncludeEmptyDirectories { get; set; } // archives; default true
public bool PreserveTimestamps { get; set; } // archives; default true
public string Password { get; set; } // archives; default null
public bool OverwriteExisting { get; set; } // archives; default false
public string FileFilter { get; set; } // archives; default null
public int MaxDegreeOfParallelism { get; set; } // default Environment.ProcessorCount
public bool UseParallelProcessing { get; set; } // default true
public static CompressionSettings Default { get; } // GZip, Normal, 64 KB
public static CompressionSettings Fast { get; } // GZip, Fastest, 256 KB
public static CompressionSettings MaxCompression { get; } // GZip, Maximum, 64 KB
public static CompressionSettings ZipArchive { get; } // ZIP, Normal (SCYLLA_HAS_SHARPZIPLIB)
public CompressionSettings Clone();
public void Validate(); // throws CompressionException on invalid values
}
public enum CompressionAlgorithm { GZip, Deflate, Zip, BZip2 } // Zip and BZip2 require SCYLLA_HAS_SHARPZIPLIB
public enum CompressionLevel { None, Fastest, Normal, Maximum }
public enum CompressionErrorCode
{
Unknown, NullData, CorruptedData, UnsupportedAlgorithm,
OperationCancelled, FileNotFound, AccessDenied, FileExists,
InvalidSettings, EntryNotFound, InvalidPassword, WrongPassword,
InsufficientSpace, IOError,
}
public readonly struct CompressionProgress
{
public long BytesProcessed { get; }
public long? TotalBytes { get; }
public double PercentComplete{ get; } // -1 when total is unknown
public string CurrentFile { get; } // archive operations only
public int FilesProcessed { get; }
public int? TotalFiles { get; }
}
public readonly struct ArchiveEntryInfo
{
public string Name { get; }
public string FullPath { get; }
public long UncompressedSize { get; }
public long CompressedSize { get; }
public bool IsDirectory { get; }
public DateTime LastModified { get; }
public bool IsEncrypted { get; }
public double CompressionRatio { get; } // 0..100, or -1 for directories
}
public sealed class CompressionException : Exception
{
public CompressionErrorCode ErrorCode { get; }
}
public readonly struct CompressionResult { /* result-pattern wrapper for void ops */ }
public readonly struct CompressionResult<T> { /* result-pattern wrapper carrying T */ }
See the API reference for full signatures, remarks, and exception contracts on every member.
Settings reference
CompressionSettings bundles every parameter the facade needs to compress: which algorithm, what level, how big a buffer, how aggressively to parallelise, and the archive-specific knobs (password, file filter, overwrite, timestamps, empty directories). The three pre-built presets cover almost every gameplay scenario.
| Property | Default | Range / Accepted values | Effect |
|---|---|---|---|
Algorithm | CompressionAlgorithm.GZip | GZip, Deflate, Zip, BZip2 (* requires SharpZipLib) | Picks the compression algorithm. Zip is the only one that produces multi-file archives. |
Level | CompressionLevel.Normal | None, Fastest, Normal, Maximum | Trade-off between output size and CPU time. None stores without compression (still wraps in the format header). On GZip/Deflate, Normal and Maximum are equivalent. |
BufferSize | 65,536 (64 KB) | 1,024 (MIN_BUFFER_SIZE) to 16,777,216 (MAX_BUFFER_SIZE) | Stream copy buffer. Larger buffers improve throughput on large sequential reads at the cost of memory. Use LARGE_BUFFER_SIZE (256 KB) for files over 10 MB. |
IncludeEmptyDirectories | true | true / false | Archives only. When true, empty subdirectories in the source folder produce explicit directory entries in the archive. |
PreserveTimestamps | true | true / false | Archives only. When true, each entry’s LastModified is the source file’s modification time. Set false for deterministic archive bytes across rebuilds. |
Password | null | non-empty string for AES-256, null or empty for no encryption | Archives only. Triggers AES-256 entry encryption when set. Both creation and extraction must agree on the password. |
OverwriteExisting | false | true / false | Archive extraction. When false, an existing destination file raises CompressionErrorCode.FileExists; when true, the existing file is silently replaced. |
FileFilter | null | shell-style wildcard such as *.json or data_*.bin | Archive creation. Restricts which files in the source folder are packed. Case-insensitive on Windows, case-sensitive on Unix-likes. |
MaxDegreeOfParallelism | Environment.ProcessorCount | >= 1 | Batch and archive parallelism cap. Set to 1 to force sequential processing regardless of UseParallelProcessing. |
UseParallelProcessing | true | true / false | Master switch for batch and archive parallelism. Set to false for deterministic single-threaded processing (debugging, reproducibility tests). |
Presets (shared, immutable; call Clone() to derive a mutable copy):
| Preset | Algorithm | Level | Buffer | Available with | Best for |
|---|---|---|---|---|---|
Default | GZip | Normal | 64 KB | Always | The recommended choice for most game data: save files, exports, JSON. |
Fast | GZip | Fastest | 256 KB | Always | Runtime hot paths: network payloads, incremental save snapshots, streaming. |
MaxCompression | GZip | Maximum | 64 KB | Always | Offline asset packing, archival, release builds where output size matters. |
ZipArchive | Zip | Normal | 64 KB | SCYLLA_HAS_SHARPZIPLIB | Multi-file archive operations. Clone and set Password for encrypted ZIPs. |
Error codes
CompressionException.ErrorCode is the canonical way to programmatically distinguish failures. The codes below cover every documented path.
| Code | When it appears |
|---|---|
Unknown | Fallback for an unspecified failure. Also the neutral sentinel on successful CompressionResult instances; always check IsSuccess before using. |
NullData | A required byte-array, stream, or path parameter was null where non-null was required. |
CorruptedData | The compressed input is truncated, malformed, or uses an unrecognised format. Often means “wrong algorithm” or “not actually compressed data”. |
UnsupportedAlgorithm | CompressionAlgorithm.Zip or BZip2 requested without the com.unity.sharp-zip-lib package installed. |
OperationCancelled | The async or progress-bearing operation was cancelled via its CancellationToken. Partial output is discarded. |
FileNotFound | A source file or directory path does not exist. Common when extracting or compressing a missing input. |
AccessDenied | OS permission denied for reading the source or writing the destination. Check file permissions or run with adequate privileges. |
FileExists | The destination already exists and CompressionSettings.OverwriteExisting is false. Enable overwrite or delete the existing file first. |
InvalidSettings | CompressionSettings.Validate() rejected one or more property values (buffer size out of range, parallelism less than one). Validate up front. |
EntryNotFound | ExtractEntry referenced a path that does not exist inside the archive. Use ListArchiveEntries to discover the available paths first. |
InvalidPassword | An encrypted-archive operation requires a password but CompressionSettings.Password is null or empty. |
WrongPassword | The supplied archive password does not match the one used to create the archive. Prompt the player to retry. |
InsufficientSpace | The target volume does not have enough free space to complete the operation. |
IOError | Generic I/O failure (network drop on a network drive, disk error, etc.). Inspect CompressionException.InnerException for the underlying IOException. |
Best Practices
- Default to GZip unless there is a specific reason otherwise. It is always available, widely supported, self-describing, and produces good ratios at reasonable speed. Switch only when a measured need (multi-file archives -> ZIP; smaller text-heavy outputs at low frequency -> BZip2) justifies the move.
- Pick
CompressionSettings.Defaultfor offline writes,Fastfor runtime hot paths,MaxCompressionfor cold archives. Each preset is tuned for a different point on the latency-vs-size curve. Custom settings are rarely better than the matching preset for a common workload. - Reach for the stream API for any input larger than a few megabytes. The byte-array overloads allocate the full output up front, which pressures the LOH and visibly hitches on mobile. The stream API holds one buffer at a time.
- Use
Asyncvariants for any user-visible compression. AMaxCompressionGZip on tens of megabytes can take hundreds of milliseconds; running it on the main thread produces a hitch. Async offloads it to a worker thread so your game keeps rendering. - Always call
Validate()on a customCompressionSettingsinstance before passing it to a compression method. The facade callsValidateinternally, but checking at the call site surfaces configuration errors next to the configuration, not deep in the provider. - Reach for
TryDecompresswhen the input might be wrong. User-supplied files, possibly corrupted network blobs, and any untrusted source are well served by theTryvariant; reservetry/catchfor paths where decompression is expected to succeed. - Inspect
CompressionException.ErrorCoderather than parsing exception messages. Codes are stable across releases; messages are not, and they vary across underlying providers. - Compress first, encrypt second. Encrypted bytes look random and yield zero compression. The combined
CompressAndEncrypthelpers enforce the correct order; prefer them over a hand-wired chain. - Use
CompressBatchfor many independent buffers. A loop of single-bufferCompresscalls is single-threaded;CompressBatchparallelises acrossMaxDegreeOfParallelismworker threads automatically. - Set
ScyllaCompression.DefaultAlgorithmandDefaultLevelonce at startup if non-default values are needed. The static properties are not synchronized; set them before spawning threads that call compression. - For files that cross platforms, prefer GZip or ZIP. They are recognised by virtually every operating system, every command-line tool, and every browser. Raw Deflate and BZip2 are less universally supported.
- Use
IsAlgorithmSupportedbefore requesting Zip or BZip2 in code that may run without SharpZipLib. A direct request on a build without the define raisesUnsupportedAlgorithm; the check avoids the exception in the first place.
Pitfalls
Related
- Crypto Utils
- Serialization Utils
- File IO Utils
- API reference:
Scylla.Core.Util.Compression.ScyllaCompression - API reference:
Scylla.Core.Util.Compression.CompressionExtensions - API reference:
Scylla.Core.Util.Compression.CompressionSettings - API reference:
Scylla.Core.Util.Compression.CompressionAlgorithm - API reference:
Scylla.Core.Util.Compression.CompressionLevel - API reference:
Scylla.Core.Util.Compression.CompressionProgress - API reference:
Scylla.Core.Util.Compression.CompressionException - API reference:
Scylla.Core.Util.Compression.CompressionErrorCode - API reference:
Scylla.Core.Util.Compression.ArchiveEntryInfo