Subsampled views: pull from a share of the cells, and reproduce it exactly#

Open In Colab

A dataset of millions of cells is often opened with a share of them: Cellucid opens a seeded subsample when every cell would not fit the device, and you can choose one under Session › Subsampling › Cells to open (Subsampled loading). Everything you make then covers only the open cells. This tutorial shows what pull writes for the cells that were not open, and how to take exactly the open cells in Python with cellucid.subsample_cells().

Time: 10 minutes. The same tutorial in R: Subsampled views in R.

# A fresh runtime (Google Colab, a new environment) installs Cellucid once.
import importlib.util

if importlib.util.find_spec("cellucid") is None:
    %pip install --quiet cellucid

The data#

The 2,531-cell mouse pancreas (endocrinogenesis, day 15.5) AnnData checked into the cellucid-python repository: UMAP in obsm['X_umap_2d'], cell types in obs['clusters'], 27,998 genes. In a clone of the repository the cell below finds it; anywhere else (Colab) it downloads it once. Use your own adata in its place.

from pathlib import Path
from urllib.request import urlretrieve

import anndata as ad

import cellucid

NAME = "endocrinogenesis_day15.5_raw.h5ad"
URL = (
    "https://raw.githubusercontent.com/theislab/cellucid-python/main/"
    "docs/user_guide/python_package/f_notebooks_tutorials/datasets/" + NAME
)


def pancreas_h5ad() -> Path:
    """The Pancreas file of a cellucid-python clone, else a copy downloaded once (35 MB)."""
    here = Path.cwd().resolve()
    for base in (here, *here.parents):
        for candidate in (base / "datasets" / NAME, base / "docs/user_guide/python_package/f_notebooks_tutorials/datasets" / NAME):
            if candidate.is_file():
                return candidate
    if not (here / NAME).is_file():
        urlretrieve(URL, here / NAME)
    return here / NAME


DATA_PATH = pancreas_h5ad()
adata = ad.read_h5ad(DATA_PATH)
adata

DATASET_ID = "pancreas-e15-5"

viewer = cellucid.show_anndata(DATA_PATH, dataset_name="Pancreas E15.5", dataset_id=DATASET_ID)

Open a share of the cells#

In the viewer, expand Session and its Subsampling; under Cells to open, set the slider to 20 %, choose Uniform as the method, leave the seed at 0, and press Apply. The view now shows 506 of its 2,531 cells (the Dataset box says “506 of 2.5K cells”, and its Details “506 of 2,531 (uniform, seed 0)”): the first 506 of their priority order, kept in place where they were. Uniform with seed 0 is also what File default (uniform) chooses here: Python serves the file with its cells in priority order under seed 0, so the share is the first rows of every array and only those cells are sent.

Then lasso some cells and Confirm, and run the next cell.

import numpy as np

opened = next(result for result in viewer.results if result.kind == "subset")
open_cells = np.arange(adata.n_obs) if opened.cells is None else opened.cells.astype(np.int64)
print(f"{open_cells.size} of {adata.n_obs} cells are open; digest {opened.meta['digest']}")

Every result names cells by their row in the whole dataset (their original position), whatever share is open, and one result, of kind subset, lists the open cells themselves. Its digest is the one the Session panel shows (“Cells …”).

Pull: unknown, not “not selected”#

summary = viewer.pull(adata)
lasso = next(name for name in summary.columns if name.startswith("cellucid_highlight__"))
print(adata.obs[lasso].dtype)
print(adata.obs[lasso].value_counts(dropna=False))

A highlight column of a subsampled viewer has the nullable boolean dtype: True for the highlighted cells, False for open cells that are not, and <NA> for the cells the viewer never opened — nobody looked at them, so they are unknown. A label or score column is missing (NaN) there. Count and test on the open cells only:

on_open = adata.obs[lasso].iloc[open_cells]
print(int(on_open.sum()), "of", on_open.size, "open cells highlighted")

Reproduce the subsample exactly#

cellucid.subsample_cells() follows the same specification as the web app (and R’s cellucid_subsample_cells()), checked against one shared conformance fixture: the same data, method, size and seed give the same cells.

kept = cellucid.subsample_cells(adata, fraction=0.2, method="uniform", seed=0)
print(kept.size, np.array_equal(kept, open_cells))
small = adata[kept].copy()  # exactly the cells the viewer showed
small

The other methods take what they need, as the panel shows it:

In the viewer

In Python

Blocks (the default for a file Cellucid did not write, opened by a link or from disk; the Dataset box’s Details say “blocks of …”)

subsample_cells(adata, fraction=0.2, method="blocks", block_size=..., seed=0)

Stratified on clusters

subsample_cells(adata, fraction=0.2, method="stratified", groupby="clusters")

Stratified, Same count per category

... method="stratified", groupby="clusters", allocation="balanced"

Spatial on X_umap_2d

subsample_cells(adata, fraction=0.2, method="spatial", basis="X_umap_2d")

With uniform and blocks, a smaller share keeps a subset of the cells a larger one keeps (same seed), so moving the slider up keeps the cells you already saw:

ten = cellucid.subsample_cells(adata, fraction=0.1, seed=0)
print(np.isin(ten, kept).all())
balanced = cellucid.subsample_cells(adata, 350, method="stratified", groupby="clusters", allocation="balanced")
adata.obs["clusters"].iloc[balanced].value_counts()

Share the same cells#

A link can name the subsample, so everyone who opens it sees the same cells (the link names how they are chosen, never the cells):

https://www.cellucid.com/?data=<your dataset>&cells=20%&sample=uniform&seed=0

For a dataset opened by a link, Copy link under Cells to open makes one (a dataset a notebook serves has no link to share). A session saved on a subsample records its cells too: cellucid.pull(adata, "my.cellucid-session") writes its highlights with <NA> for the cells it did not open, as above.

viewer.stop()

Next#