Bioinformatics Benchmarks: SLAF vs Traditional Formats¶

SLAF provides dramatic performance improvements over traditional single-cell data formats for common bioinformatics operations. This document presents comprehensive benchmarks comparing SLAF against h5ad (AnnData) and TileDB SOMA across realistic bioinformatics workflows.

Overview¶

These benchmarks demonstrate SLAF's performance advantages in three key areas:

1. Metadata Filtering - Cell and gene filtering operations
2. Expression Queries - Retrieving expression data for specific cells/genes
3. Preprocessing Pipelines - Scanpy-based preprocessing workflows

Test Dataset: synthetic_50k_processed¶

• Cells: 49,955 cells
• Genes: 25,000 genes
• Input File Size: ~722MB h5ad file

Hardware Configuration¶

• Machine: Apple MacBook Pro with M1 Max
• Memory: 32 GB RAM
• Storage: 1 TB NVMe SSD (local disk)
• OS: macOS 13.6.1
• Python: 3.12.0

Cell Filtering Benchmarks¶

Cell filtering is a fundamental operation in single-cell analysis, used for quality control, cell type selection, and data subsetting.

Performance Results¶

Average Performance:

• SLAF vs h5ad: 92.3x faster
• SLAF vs TileDB: 9.4x faster
• Memory Usage: SLAF uses 115.7x less memory than h5ad

Key Insights¶

Gene Filtering Benchmarks¶

Gene filtering operations are essential for feature selection, quality control, and differential expression in single-cell analysis.

Performance Results¶

Average Performance:

• SLAF vs h5ad: 17.3x faster
• SLAF vs TileDB: 8.6x faster
• Memory Usage: SLAF uses 2.2x less memory than h5ad

Key Insights¶

Expression Queries Benchmarks¶

Expression queries retrieve specific expression data for cells or genes, supporting analysis workflows that require targeted data access.

Performance Results¶

Average Performance:

• SLAF vs h5ad: 9.5x faster
• SLAF vs TileDB: 0.9x faster
• Memory Usage: SLAF uses 154.6x less memory than h5ad

Key Insights¶

Lazy Computation (Preprocessing Pipelines)¶

SLAF enables lazy computation graphs that build complex preprocessing pipelines and only execute them on the slice of interest, similar to Dask's delayed computation patterns.

Traditional Approach (Eager Processing)¶

# Each step loads data into memory
adata = sc.read_h5ad("data.h5ad", backed="r")

# QC metrics calculation
sc.pp.calculate_qc_metrics(adata, inplace=True)

# Cell filtering with min_counts and min_genes
sc.pp.filter_cells(adata, min_counts=500, min_genes=200, inplace=True)

# Gene filtering
sc.pp.filter_genes(adata, min_counts=10, min_cells=5, inplace=True)

# Normalization
sc.pp.normalize_total(adata, target_sum=1e4, inplace=True)

# Log transformation
sc.pp.log1p(adata)

# Each operation processes the entire dataset
expression = adata.X[cell_ids, gene_ids]

SLAF Approach (Lazy Computation Graph)¶

# Build lazy computation graph
adata = LazyAnnData("data.slaf")  # LazyAnnData object

# QC metrics calculation (lazy)
pp.calculate_qc_metrics(adata, inplace=True)

# Cell filtering (lazy)
pp.filter_cells(adata, min_counts=500, min_genes=200, inplace=True)

# Gene filtering (lazy)
pp.filter_genes(adata, min_counts=10, min_cells=5, inplace=True)

# Normalization (lazy)
pp.normalize_total(adata, target_sum=1e4, inplace=True)

# Log transformation (lazy)
pp.log1p(adata)

# Only execute the computation on the slice of interest
expression = adata.X[cell_ids, gene_ids].compute()  # LazyExpressionMatrix.compute()

Performance Results¶

Key Insight: Lazy computation enables complex preprocessing pipelines that would cause memory explosions with traditional tools. The computation cost is paid when materializing results, but the memory efficiency enables workflows impossible with eager processing. This is similar to Dask's delayed computation patterns.

For detailed migration guides, see SLAF vs h5ad Benchmarks and SLAF vs TileDB Benchmarks.