Prepare a real Pancreas dataset for Cellucid

This notebook turns the checked-in pancreatic endocrinogenesis AnnData into a compact Cellucid-ready AnnData: a standard .h5ad laid out for fast browser reads, naming itself in uns['cellucid']. It is intentionally small enough to run on a laptop and contains no private paths or hidden input files.

The example writes the real 2D UMAP and the PCA latent space, every observation field, the connectivity graph, and 128 highly variable genes. That keeps the tutorial quick while exercising the same prepare(...) contract used for larger datasets.

See Prepare a Cellucid-ready AnnData for a detailed explanation of every argument.

import shutil
import tempfile
from pathlib import Path

import anndata as ad

from cellucid import prepare, show_anndata

DATASET_RELATIVE_PATH = Path(
    "docs/user_guide/python_package/f_notebooks_tutorials/"
    "datasets/endocrinogenesis_day15.5_raw.h5ad"
)


def find_pancreas_h5ad(start: Path) -> Path:
    """Find the checked-in tutorial input from any directory inside the repo."""
    for base in (start.resolve(), *start.resolve().parents):
        candidate = base / DATASET_RELATIVE_PATH
        if candidate.is_file():
            return candidate
        candidate = base / "datasets" / DATASET_RELATIVE_PATH.name
        if candidate.is_file():
            return candidate
    raise FileNotFoundError(
        f"Could not find the checked-in Pancreas tutorial file {DATASET_RELATIVE_PATH}. "
        "Run this notebook from a clone of the cellucid-python repository."
    )


DATA_PATH = find_pancreas_h5ad(Path.cwd())
WORK_DIR = Path(tempfile.mkdtemp(prefix="cellucid-pancreas-tutorial-"))
OUTPUT = WORK_DIR / "pancreas-day15-5.h5ad"

print("Input:", DATA_PATH)
print("Output:", OUTPUT)

adata = ad.read_h5ad(DATA_PATH)

assert "X_umap_2d" in adata.obsm
assert "X_pca" in adata.obsm
assert "connectivities" in adata.obsp
assert "highly_variable_genes" in adata.var

highly_variable = adata.var["highly_variable_genes"].eq("True")
genes = adata.var_names[highly_variable][:128]
adata_genes = adata[:, genes].copy()

print(adata)
print(f"Preparing {adata_genes.n_obs:,} cells and {adata_genes.n_vars:,} genes")

Write the Cellucid-ready AnnData

prepare() checks every input, then writes one .h5ad: the expression matrix gene-major (CSC), arrays chunked for HTTP range reads, and the dataset’s id, name, source and launch defaults in uns['cellucid']. The browser then reads one gene with a few small requests.

path = prepare(
    OUTPUT,
    format="h5ad",
    embeddings={"X_umap_2d": adata.obsm["X_umap_2d"], "X_pca": adata.obsm["X_pca"]},
    dataset_id="pancreas-endocrinogenesis-day15-5-tutorial",
    dataset_name="Pancreatic endocrinogenesis (day 15.5)",
    obs=adata.obs,
    var=adata_genes.var,
    gene_expression=adata_genes.X,
    connectivities=adata.obsp["connectivities"],
    latent_key="X_pca",
    obs_keys=["clusters", "clusters_coarse", "day", "phase", "palantir_pseudotime"],
    dataset_description="A compact, real Pancreas dataset prepared by the Cellucid tutorial.",
    source_name="scVelo Pancreas vignette",
    source_url="https://scvelo.readthedocs.io/en/stable/vignettes/Pancreas.html",
)

print("Cellucid-ready AnnData written to", path)

prepared = ad.read_h5ad(path)

assert prepared.uns["cellucid"]["id"] == "pancreas-endocrinogenesis-day15-5-tutorial"
assert prepared.X.format == "csc"
assert prepared.n_vars == adata_genes.n_vars

print(prepared)
print("uns['cellucid']:", prepared.uns["cellucid"])

Open the prepared result

The file names itself, so show_anndata needs no dataset_name or dataset_id. The same file opens with cellucid serve, with the web app’s Open… button, or from a link once hosted.

viewer = show_anndata(path, height=650)
viewer

Clean up

Run this after you finish exploring. To keep the file, copy OUTPUT elsewhere before cleanup.

viewer.stop()
shutil.rmtree(WORK_DIR)
print("Temporary output removed.")