Share Without Running a Server (Publish a Dataset Catalog)#

This tutorial shows how to publish AnnData files so anyone can open them in Cellucid from one link, without running a server.

You’ll learn two closely related workflows:

  1. Public catalog (recommended): a GitHub repository or any static host

    • You publish a folder holding your AnnData files and a small datasets.json.

    • Anyone opens Cellucid with ?data=<that folder> and picks a dataset.

  2. Your own sample list (optional): run or deploy the web app yourself

    • You point the web app’s Sample dataset list at your catalog.

    • Useful for offline demos, an intranet, or your own deployment.

To publish a single file, you need no catalog at all: a link to the .h5ad, .zarr.zip or Zarr store folder opens it (Open from a link). If you just want to open your own local files right now, use Browser File Picker (No Python Setup). For a complete, inspectable public repository with five tiny datasets, use Publish a Custom Dataset Repository.

At A Glance#

Audience

  • Wet lab / beginner: follow the checklist; you do not need to understand file formats.

  • Computational users: pay attention to dataset ids, file layout and host limits.

  • Developers: this is the exact datasets.json contract the web app reads.

Time

  • One dataset in a public repository: ~15–30 minutes (mostly uploading)

Prerequisites

  • Python with cellucid installed (or R with cellucid), to write Cellucid-ready AnnData files; any AnnData file works, too

  • For a public catalog: a GitHub account or a static host that allows cross-origin reads (CORS)

Important: Privacy and Sharing#

Publishing files on a public host publishes your dataset.

Before you do this:

  • Verify there is no private metadata in obs (e.g. patient ids, internal notes).

  • Remove or rename columns that should not be public; prepare(obs_keys=...) writes only the columns you name.

If you need private sharing:

  • Use Server Mode (Server Mode (CLI + Python)) on a private machine + VPN/SSH tunnel.

  • Or send the file and let each collaborator open it with Open….

The Catalog Folder Layout#

A dataset catalog is a folder holding datasets.json and the AnnData files it lists:

catalog/                         # the folder you link to
├── datasets.json                # required
├── pbmc_demo.zarr/              # a Zarr store folder (recommended on static hosts)
├── lung_atlas.h5ad              # an .h5ad file
├── organoid.zarr.zip            # a zipped Zarr store
└── pbmc_demo.cellucid-session   # optional: a dataset's default view

Key rules

  • datasets.json sits in the folder you link to.

  • Every path in it is relative to that folder, and stays below it.

  • Files can sit in subfolders (data/pbmc_demo.zarr/).

Step 1 — Write Cellucid-ready AnnData Files#

Any AnnData file opens, but a file written by cellucid.prepare() (Python) or cellucid_prepare() (R) reads fastest over a network and carries its own name, description, citation and launch defaults in uns['cellucid'].

import cellucid

cellucid.prepare(
    "catalog/pbmc_demo.zarr",
    format="zarr",
    embeddings={"X_umap": adata.obsm["X_umap"], "X_pca": adata.obsm["X_pca"]},
    obs=adata.obs,
    var=adata.var,
    gene_expression=adata.X,
    connectivities=adata.obsp["connectivities"],
    dataset_id="pbmc_demo",
    dataset_name="PBMC demo",
    dataset_description="PBMC scRNA-seq",
    latent_key="X_pca",
)

The arguments you should understand, even as a beginner:

  • dataset_id: a stable identifier; sessions and community annotations are keyed by it. 1–180 ASCII letters, digits, ., _ or -, starting with a letter or digit and not ending with .. Avoid changing it after you publish (Dataset identity (why it matters)).

  • dataset_name / dataset_description: what the dataset list and the Dataset box show.

  • format (required; the path ends in its suffix): "zarr" is a folder of many small files, for GitHub (no file over 100 MB, no LFS); "h5ad" is one file, for single-file hosts (Zenodo, Figshare, Hugging Face, S3, a lab server), local files, cellucid serve and Jupyter; "zarr.zip" is the Zarr store in one file, wherever an .h5ad goes (Which format?).

  • vector_fields (optional): per-cell vectors drawn by the velocity overlay (Vector Field / Velocity Overlay (GPU Particle Overlay)).

Every argument: Preparing an AnnData for Cellucid.

Step 2 — Write datasets.json#

datasets.json names the datasets, where each file is, and which one opens first:

{
  "schema": 2,
  "default": "pbmc_demo",
  "datasets": [
    { "id": "pbmc_demo", "path": "pbmc_demo.zarr/" },
    { "id": "lung_atlas", "path": "lung_atlas.h5ad" }
  ]
}

  • schema is the integer 2.

  • default is the id of the dataset the catalog opens on.

  • Each id is the dataset’s stable id. When a file states its own id in uns['cellucid']['id'] (every file prepare() writes does), the catalog must list it under that same id, or the file is refused when it is opened.

  • Each path ends in / for a Zarr store folder, .h5ad for an HDF5 file, or .zarr.zip for a zipped store; a file in any other format Cellucid reads (.h5mu, .loom, .h5, .rds, …) ends in its own ending. Write each part in its canonical URL encoding (cell%20atlas.h5ad), without ., .., a query or a fragment.

The dataset list shows each dataset’s name and cell count, read from its file: names, descriptions and citations are never written in the catalog. While the dataset Cellucid opens first is still loading, the list already offers every entry under the name its path gives it (suo for suo.zarr/), the one being opened marked · opening…; the names and counts follow once the files have been read. The full contract, including the optional default session, is in File format expectations.

Step 3 — Validate Locally (Before You Upload)#

  • One file: open it with Open… under File or folder (File… for an .h5ad, a .zarr.zip or another file, Folder… for a store folder). If it opens there, it will open from a link.

  • The whole catalog: serve the folder with any static server that sends CORS headers, then open ?data=http://127.0.0.1:8000/ in Cellucid:

    cd catalog
python -m http.server 8000   # add CORS headers if Cellucid runs on another origin

This avoids debugging “hosting problems” that are actually file problems.

The whole Cellucid window with the Pancreas sample coloured by cell_type: the sidebar shows Coloring and filtering with the categorical field selected and its eight-row legend, and the 3D embedding fills the canvas in the matching colours.

A loaded dataset in Cellucid: the sidebar controls the active view while the categorical legend maps directly to the colored points.#

Step 4 — Publish to a Public GitHub Repository#

4.1 Create a repository#

A data-only repository is simplest, e.g. my-lab/cellucid-data, with the catalog folder at its root or in a folder such as catalog/.

4.2 GitHub constraints (do not skip)#

  • Public repository only: Cellucid reads files through raw.githubusercontent.com.

  • Branches: shorthand without a branch reads the repository’s default branch (whatever it is called); Cellucid never probes branch names. Name another branch explicitly: owner/repo@my-branch/catalog, or https://github.com/owner/repo/tree/my-branch/catalog.

  • File size: GitHub refuses files over 100 MiB. A Zarr store folder is many small chunk files, so it fits where a large .h5ad does not; see Host a Dataset on GitHub for the steps, the limits and the Git LFS trade-off.

4.3 What to enter in Cellucid#

Expand Session, type the path into the Link or server bar under Open data, and press Open (or Enter):

  • owner/repo (if datasets.json is at the repository root)

  • owner/repo/catalog (if your catalog is in a folder)

  • owner/repo@my-branch/catalog (on another branch)

The Link or server bar with the GitHub path "theislab/cellucid-demo-custom-datasets/catalog" typed into it and the pointer on its Open button.

The Link or server bar takes the catalog folder (or one file in it). Press Open beside it.#

Cellucid then reads datasets.json, describes every listed file from its metadata (name, cell and gene counts), lists them in the dataset dropdown (labelled “Dataset from ” while the catalog is connected), and opens the catalog’s default dataset; each other dataset is opened when you choose it. A line under the bar reads Link with the address and Disconnect, which brings the samples back.

Shareable URL

https://www.cellucid.com/?data=gh:owner/repo/catalog&dataset=pbmc_demo

Optional: Your Own Sample Dataset List#

The Sample dataset list reads the catalog named by the page’s <meta name="cellucid-sample-catalog" content="..."> element in cellucid/index.html (www.cellucid.com names https://theislab.github.io/cellucid-datasets/anndata/). One sampleCatalog query field replaces it for a single launch:

http://localhost:8000/?sampleCatalog=http://localhost:9000/catalog/

The value is an absolute http(s) folder address ending in /. This is a development and deployment hook; to share a catalog with others, use a data link, ?data=https://host/catalog/ (Open from a link).

Note

A catalog on another origin must allow the Cellucid origin with CORS headers, including range requests (Range request header; Content-Range and Content-Length exposed), so the browser can read the files in place.

Edge Cases#

  • Changed dataset ids after publishing: links break; saved sessions and community annotations no longer match.

  • A file whose uns['cellucid']['id'] differs from its catalog id: the file is refused when chosen; list it under the id it states.

  • Wrong repository path or branch: Cellucid reports that the folder holds no datasets.json, naming the address it read.

  • A Zarr store without consolidated metadata on a static host: write it with prepare() or call zarr.consolidate_metadata() before publishing.

  • GitHub Pages and hidden files: Jekyll skips dot-files such as .zmetadata and .zgroup; publish a .nojekyll file at the site root.

Troubleshooting (Large, Practical)#

Symptom: “Invalid datasets.json: the catalog must contain exactly: datasets, default, schema (missing: schema)”#

The catalog has no "schema" key. A catalog holds exactly "schema": 2, "default" and "datasets", and lists the files it serves (AnnData written with prepare() reads fastest).

Symptom: “The dataset list shows up, but choosing a dataset fails”#

Likely causes

  • A path names a file or folder that does not exist (404).

  • The file states another id than its catalog entry.

  • The file is not in a format Cellucid reads, or a Zarr store lacks its metadata.

Fix

  • Compare each path with the published names; the notification names the dataset and the reason.

Symptom: “It loads, but gene search is slow”#

Likely causes

  • The expression matrix is stored cell-major (CSR): the first gene waits for its index to be built.

Fix

  • Write the file with prepare(), which stores the matrix gene-major (CSC).

Symptom: “The vector field overlay toggle is disabled / no fields appear”#

Likely causes

  1. The file has no obsm['<field>_<basis>'] array beside its embedding obsm['X_<basis>'] (velocity_umap beside X_umap), with the same number of columns.

  2. The field is bound to another embedding than the one the view shows.

Fix

  • Write the vectors with prepare(..., vector_fields=...), or name them <field>_<basis> in obsm after their embedding X_<basis>.

  • Show an embedding the field is bound to; the overlay info line names them.

See Troubleshooting (velocity overlay).

Symptom: “GitHub blocks my push / files are too big”#

Fix

  • Write the dataset as a .zarr folder: each chunk is a small file.

  • Or host the files elsewhere (Zenodo, Hugging Face, an object store) and link to them, or use server mode.

Next Steps#