Sessions (.cellucid-session files)#

A Cellucid session file holds the history of what was done in the viewer: every step a person, an agent or a notebook took on a dataset (renaming a category, highlighting cells, merging categories, pushing a column, saving a gene set…), grouped in trackers (independent lines of work, each a tree of steps with named branches), with notes, bookmarks, turns, the journal of the history’s own edits, and the state every step left behind. Python reads it, and applies the state at any step to an AnnData: a plan first, then the change.

  • File extension: .cellucid-session (session format 4)

  • Use cases:

    • “I selected and annotated cells in the UI; now I want those labels back in Python.”

    • “I want the labels as they were before that last merge, or on another branch.”

    • “I want a reproducible trail of what was done, and by whom.”

This page documents:

Audience + prerequisites#

Audience

  • Wet lab / beginner: use the “Apply to AnnData” workflow and the troubleshooting section.

  • Computational: pay attention to exact dataset identity and cell ordering assumptions.

  • Developer: read the “Format + safety” section before handling session files from untrusted sources.

Prerequisites

  • If applying to AnnData: an AnnData (or MuData) object. Pandas is part of the Cellucid core installation.

  • A session file, saved from the web app (Share…) or asked of a notebook viewer.

Fast path (apply what was done in the viewer to an AnnData)#

import cellucid

session = cellucid.read_session("my-session.cellucid-session")

plan = session.apply(adata)      # what applying the step the viewer showed would change
plan.table                       # one row per change: kind, domain, name, column, from, cells, status, note
report = plan.apply()            # writes the rows into adata, all at once, and records the apply

Now inspect:

  • adata.obs: renamed columns and categories, one boolean column per highlight group, one column per user-defined field.

  • adata.uns["cellucid"]["gene_sets"]: the gene sets of the step.

  • cellucid.history_applied(adata): every apply recorded on adata.

Practical path (common workflows)#

1) How to obtain a session file#

A) From the web app (download a .cellucid-session file)#

Use this when you explore in the standalone browser viewer (notebook or not): Share… writes the dataset’s history (the trackers and branches ticked), the current state only, or the steps only to one file. A file of the steps only holds no values: it can be read, not applied.

The State section with the Pancreatic endocrinogenesis sample open: Session with Share… and Load…, and the History with the recording switch, the branch shown and the graph of the steps taken so far, the step shown marked.

The State section of the sidebar keeps Share… and Load… under Session, above the history they write and read.#

B) From a notebook viewer without downloading#

A notebook or browser-tab viewer hands Python the session directly (viewer.request_session(), then result(); Jupyter (notebook embedding + hooks)). To bring results onto an AnnData without a session, viewer.pull(adata) (Round trip (results back from the viewer, columns and gene sets into it)).

2) Read the history#

session = cellucid.read_session("my-session.cellucid-session")

session.dataset_fingerprint   # dataset id, name, counts, cell order, data mapping
session.source                # where it was opened from, or None
session.requires              # Requires(obs=(...), var=(...)): the fields the shown state uses
session.trackers              # every tracker, in the file's order

tracker = session.tracker()                   # the tracker the viewer showed
tracker = session.tracker("Annotation")       # or one by its name or id
tracker.steps[["id", "branch", "label", "origin"]]
tracker.branch("Without score").steps         # a branch: its path from the root to its tip
tracker.notes_table()                         # every note: on the tracker, a branch, a group, a step or a turn
tracker.group("QC").steps                     # a group: a named run of consecutive steps of one branch

A tracker (cellucid.history.Tracker) holds every field of the history model’s Tracker in snake case (head_id, active_branch, main_branch, paused, start, removed, branches, groups, step_records, turns, notes, journal, chats). paused is the tracker’s recording switch: None while it records, else a cellucid.history.Pause with at (since when), by (who paused it, then everyone who acted while it was paused) and draft (always None in a file: the web app squashes what differs into one step before it saves); the changes made while paused became one squashed step when it resumed. tracker.steps is every step as a DataFrame, one row per step in step order (parents before children), with the columns cellucid.history.STEP_COLUMNS: id, branch, parent, kind (root, action, outside – a change made outside an action – or squashed), type (the action), v (the version of the action’s parameters it was recorded with), label (what it did, in a sentence), code (its line of pseudocode, as recorded), origin (user, agent, host – a Python or R notebook –, session, replay, startup or outside), agent, model, host, turn, at (a UTC timestamp), status (applied or unapplicable), stale (its result no longer matches its inputs at the head), bookmark, group (the group it is in) and n_notes.

A squashed step has no action of its own (type, params, v and code are None): it is the net change of several steps merged into one (“Merge into one step”, a squashed group or turn, the history cleared keeping the state) or of what changed while recording was paused, and step.squashed says why (cause: steps, clear, paused or share), how many steps it replaced (count, None when none was recorded), who acted in it (by) and what the replaced steps carried that it keeps (carried, a cellucid.history.Carried: the ids of their notes, now its own, and their bookmarks; empty unless the cause is steps). A squash in the middle of a branch keeps the id of the last step it replaced, so the steps and branches after it keep their parent. Its state is exact, as any step’s. A step’s code is its pseudocode as it was recorded: parameters of an earlier version (v below the action’s version in cellucid.action_catalog.json) are shown and applied as they were; the web app upgrades them only to replay them.

Call

What it returns

tracker.step("s5")

one step (cellucid.history.Step): by (who took it), params, outcome, touches, notes, bookmark, …

tracker.branch(name_or_id)

one branch (cellucid.history.Branch): main, path, steps (its path from the root), own_steps (the steps made on it); None is the active branch

tracker.group(name_or_id)

one group (cellucid.history.Group): name, notes, step_ids, first, last, branch, steps (its steps as a table); each step names its group (step.group)

tracker.turn(turn_id)

one turn: a notebook’s or an agent’s group of steps (tracker.turns lists them; viewer.turn() makes one)

tracker.path_to("s8")

the steps from the root to a step, root first

tracker.stale()

the results stale at the head: {"step", "cause", "domains"} each

tracker.methods(branch) / tracker.pseudocode(branch)

a branch (the main one when left out) as methods text, or as pseudocode, as the web app writes them: the tracker’s first note opens the methods; groups are headings in the methods and # --- QC --- … # --- end of QC --- in the pseudocode; a bookmark is # Bookmark: <name> (one a squashed step kept from a merged step adds “(on a step merged into this one)”); a squashed step is said as what it stands for (changes made with recording paused, steps merged, a fresh start, a shared state) and holds only values

session.conversations(tracker)

the agent conversations kept with a tracker, when the person chose to include them

A name is compared as the web app compares names (case and extra white space aside). An id or name the session does not hold is refused, naming the ones it does.

3) The state at any step#

step = tracker.step("s5")
state = step.state()                            # every state domain's value after step s5

state["field-overlays"]["renamedCategories"]    # [{"source", "field", "category", "name"}]
state["highlights"]["pages"]                    # pages of highlight groups, cells by original position
state["gene-sets"]["sets"]                      # [{"name", "description", "genes"}]

step.diff("s13").table                          # what differs between two steps, domain by domain

step.state() returns every state domain’s value as the web app captures it (cells, mapping, field-overlays, gene-sets, views, filters, colors, highlights, camera…). Cells are original positions (rows of obs in the order the dataset gives them), Uint32Array cells come back as uint32 arrays and Float32Array values as float32 (NaN: no value). A session shared as its steps only (session.scope.content == "steps") holds no values, and its states are refused. A session shared as the current state only ("state") holds one tracker: the dataset as opened and one squashed step, “Shared state”; it reads and applies as any session. No picture of a step is ever stored: the web app draws one from the live view when it is asked for.

4) Apply to AnnData (a plan, then the change)#

plan = cellucid.apply_cellucid_session_to_anndata(
    adata,
    "my-session.cellucid-session",
    tracker="Annotation",         # default: the tracker the viewer showed
    branch="Without score",       # default: the tracker's head on its active branch
)
plan.table
plan.writes_anything             # whether a row would write or replace
report = plan.apply()
report.applied, report.provenance

session.apply(adata, ...) is the same call with that session. With dry_run=True (the default) nothing is written: the ApplyPlan lists every change, one row each. plan.apply(), or dry_run=False, writes it into adata in place and returns the ApplyReport. step="s5" applies the state after that step (on branch, when both are given).

The rows, in domain order:

kind

What it is

column-renamed

an obs column renamed in place; a gene’s display name is recorded, never applied

categories-renamed / categories-merged

a category renamed, or renamed into one the column has: the two become one

field-deleted

a deletion, recorded and never applied: the column stays

user-field

one column per field the session made, named by its key: a categorical as category (no category: missing), a column a notebook pushed as float32 (no value: NaN), a continuous copy of another field copied from that obs column or gene (under the session’s data mapping); a field deleted in the session is not applied

gene-set

each gene set, written to adata.uns["cellucid"]["gene_sets"] as cellucid.pull writes them, and as one bool column cellucid_gene_set__<name> of the var of the gene axis the data mapping reads, True on the set’s genes (4) Gene sets → adata.uns["cellucid"]["gene_sets"] and boolean var columns)

highlight

one boolean column per group of every page, cellucid_highlight__<page>__<group> from the page’s name and the group’s label

presentation

one row per other domain the step changed (filters, colours, views, the camera, analyses): how the data was shown, reported and never applied

Each row’s status:

  • write: the object lacks it;

  • unchanged: the object holds it already. Applying the same step again writes nothing (report.applied is False) and is never a failure;

  • replace: a column or gene set an earlier apply wrote, holding other values, with overwrite=True;

  • not-applied, with the reason in note: a column or gene set the person made is never replaced, whatever overwrite says; a column or category the object does not have; a deletion; a presentation domain.

Every apply that writes something appends one record to adata.uns["cellucid"]["history_applied"], JSON text holding a list (JSON text keeps .h5ad and Zarr files writable). Read it with cellucid.history_applied(adata): each record names the session’s SHA-256, the tracker, branch and step, applied_at, the package and its version, the columns it wrote (which a later apply replaces only with overwrite=True), the renames it applied, what it did not apply, and, with keep_session=True, the session file itself (base64), so the history can be rebuilt from the object alone.

Only what the chosen step needs is checked, before anything is written: the cell count (of the subsample the session was saved on, when it was), the dataset id when adata.uns["cellucid"]["id"] names one, the data mapping’s keys and gene count, the cell order (when adata has the embedding the session names, the cell-order digest of its stored values over the session’s cells must be the session’s), and the obs columns and genes the step’s state uses (every missing one named at once). A plan that would write, rename or replace obs['cellucid_original_position'] (the priority-order column) is refused, dry runs included. Everything is then written at once, so a failed apply leaves adata as it was. plan.apply() makes the plan again first and refuses an object changed since.

The target may also be a MuData: the session is checked against what the viewer showed (the cells, and the modality its data mapping names), labels and columns apply to the global mdata.obs, and mdata.uns records the apply.

Deep path (format + safety)#

Treat session files as untrusted input#

Session files may come from collaborators, community annotation workflows, or downloads of unknown origin. read_session() reads and checks the whole file, as the web app does before it applies anything:

  • the signature and every frame length against the bytes the file holds;

  • every chunk inflated to exactly its declared size (zip-bomb protection);

  • every array against its digest, every value against its hash;

  • every record against the history model (history_model.schema.json, the web app’s, pinned by digest in the web app, cellucid-python and cellucid-r);

  • every tracker’s tree whole: one root, branches that are lines from their fork step to their tip, groups that are runs of consecutive steps holding whole turns, unique names, the head on the active branch’s path;

  • what the share left out is absent (scope: its content, notes, conversations, Recently deleted, times); a value or array no step refers to, one a step needs that the file lacks, and trailing bytes are refused.

Each refusal is one sentence starting This session file is damaged: and saying what. A file written by a newer Cellucid than this package is refused as newer, never as damaged: This session was written by a newer Cellucid (X) than this package reads (Y); update the cellucid package to read it.

Format overview (for developers)#

The .cellucid-session framing (session format 4) is:

  1. the signature CELLUCID_SESSION/4\n – the format version is part of it

  2. manifestByteLength (u32 little-endian)

  3. UTF-8 manifest JSON bytes

  4. repeated chunks: [chunkByteLength (u32 LE), chunkBytes...]

The manifest holds createdAt, app (the Cellucid version that wrote it and the SHA-256 of its action catalogue), actions (one stamp per action type and parameters version its steps use, {type, version, params}, sorted by type then version, params the digest of that version’s parameter schema), scope (what the share chose to include: the trackers and branches, content – history, state or steps –, results – include or recompute –, notes, conversations, Recently deleted, times, the dataset’s source), datasetFingerprint (datasetId, name, cellCount, varCount, cellOrder, dataMapping), source (where the dataset was opened from: a sample, a link, a server, or the name of a local file or folder; null for a notebook, or a share with the dataset’s name only), requires (the obs columns and genes the shown state uses), chrome (page chrome kept outside the history, which the web app puts back after a load: {"pressed": {<group id>: <button id>}}, the pressed button of each pressed-button group, such as the selection tool; {"pressed": {}} from a file written without a page; read-only as session.chrome, never applied), cells (a subsample’s), genes (a gene sample’s), activeTracker, trackers and chunks.

Chunk (id)

What it holds

steps (steps/<n>/<k>, eager)

a run of one tracker’s steps as JSON, in id order; a tracker lists its steps chunks, read in order as one list; before, after and reads name state values by their SHA-256 hash, params and outcome are skeletons

parts (parts/<priority>/<k>)

{<name>: <skeleton>}: sub-values stored once (below); eager for what the shown values name, else lazy

array (array/<digest>)

one typed array a value or step holds, stored once by its digest <type>:<length>:<sha256 of its little-endian bytes>: raw elements, or cell-set (a uvarint bound, then the cell-set codec’s smallest encoding); eager for what the shown values name, else lazy

values (values/<priority>/<k>)

{<hash>: <skeleton>}: eager, what the active head shows; lazy, every other value a step or a journal entry refers to; at least one eager chunk, none in a share of the steps only

journal (journal/<n>, lazy)

one tracker’s edits of the history itself (a note, a bookmark, a group, a squash, a branch locked, a step deleted)

chat (chat/<n>/<m>, lazy)

one conversation kept with a tracker, when the person chose to include them

Every eager chunk comes before any lazy one, and within a priority the kinds come in the order of the table. steps, parts and values are JSON cut into pieces of about 16 MiB of text. Each chunk is stored gzip’d (codec: "gzip") where that saves at least a sixteenth of its bytes, else as it is (codec: "none", its storedBytes equal to its uncompressedBytes).

A skeleton is a value as JSON with each typed array as {"$array": <digest>}, each number JSON cannot hold as {"$number": "NaN" | "Infinity" | "-Infinity" | "-0"}, and each object or list whose skeleton text is at least 512 characters as {"$part": <name>}, stored once in a parts chunk under the SHA-256 of its text (a part names the parts inside it). Values that share a sub-value (the gene names of every analysis result, a user field in each later value) store it once. A value’s hash is the SHA-256 of the value itself, joined, as canonical text: keys sorted by UTF-16 code units, numbers and strings written as JavaScript’s JSON.stringify writes them. A part is storage only: read_session joins each part once, never trusts its name (the joined value is checked against its own hash), and refuses a part that names itself through others, a value whose text would pass 2^29 − 24 characters (refused before it is expanded), and a part, array or value nothing refers to. A step’s parameters may name arrays and parts only the lazy chunks hold; read_session reads the whole file and joins them. The cell-set decoder is cellucid.session_codecs.decode_cell_set.

A session of another format version is refused with the web app’s sentence naming it (other_session_format_message()): This session was saved by an earlier version of Cellucid (session format 3), which this version no longer reads: a session file now holds the history of every step (session format 4). It was not opened. Open it in the version of Cellucid that saved it, or re-create the session in this version. Any other file that does not start with CELLUCID_SESSION/4\n is refused with: This file is not a Cellucid session file of format 4 (it does not start with CELLUCID_SESSION/4).

Closed value shapes#

Applying reads four state domains, each a closed shape (app/state/domains/README.md in the web app): cells ({open, genes}), field-overlays ({renamedFields, renamedCategories, deletedFields, userFields}), highlights ({active, pages}) and gene-sets ({sets}). A key this version does not know is refused before anything is written: a value it does not understand is never applied in part.

API reference#

read_session#

cellucid.read_session(path)[source]#

Read a .cellucid-session file whole and check it (CellucidSession).

Return type:

CellucidSession

Parameters:

path (str | Path)

Examples

>>> session = cellucid.read_session("annotation.cellucid-session")
>>> session.tracker().steps.head()

CellucidSession#

class cellucid.CellucidSession(path)[source]#

Bases: object

A .cellucid-session file (session format 4): its trackers, branches, steps and states.

Making one reads the whole file and checks all of it (see the module notes); a file of another session format, or one that breaks the format, raises ValueError with a sentence that says why (a file written by a newer Cellucid than this package says so, never that it is damaged), and nothing about it is read lazily later. read_session() makes one.

Every record is the history model’s (cellucid.history): the session’s header (created_at, app, actions, scope, dataset_fingerprint, source, requires, chrome, cells, genes, active_tracker) and its trackers, each a Tracker with its branches, its steps (a table, and each a Step), its turns, notes and journal.

Parameters:

path (str or Path) – The .cellucid-session file; ~ is expanded. A missing file raises FileNotFoundError.

Examples

>>> session = cellucid.read_session("annotation.cellucid-session")
>>> tracker = session.tracker()          # the tracker the viewer showed
>>> tracker.steps[["id", "branch", "label", "origin"]]
>>> tracker.branch("Main").steps         # the path to its tip
>>> tracker.step("s5").state()["gene-sets"]
>>> plan = session.apply(adata)          # what applying the head would change
>>> report = plan.apply()
__init__(path)[source]#
Parameters:

path (str | Path)

Return type:

None

created_at: str#

Canonical ISO time of the save (UTC).

app: AppStamp#

the Cellucid version and the SHA-256 of its action catalogue.

Type:

What wrote the file

actions: tuple[ActionStamp, ...]#

The action types its steps use, each with the digest of its parameter schema, sorted by type.

scope: Scope#

What the share or save chose to include (trackers, branches, content, results, notes, chats, journal, times, source).

active_tracker: str#

The id of the tracker the session shows.

trackers: tuple[Tracker, ...]#

Every tracker, in the file’s order (one in Recently deleted has removed set).

property sha256: str#

The SHA-256 of the file’s bytes (what an apply records as session_sha256).

property manifest: dict[str, Any]#

The manifest as the file holds it (a copy).

property dataset_fingerprint: dict[str, Any]#

datasetId, name, cellCount (the cells open), varCount, cellOrder and dataMapping (a copy).

Type:

The dataset the session was saved on

property source: dict[str, str] | None#

Where the session’s dataset was opened from, or None.

{"kind": "sample", "dataset"}, {"kind": "link", "locator", "dataset"} (locator is the ?data= value), {"kind": "remote", "url", "dataset"} or {"kind": "file", "name", "dataset"} (the local file or folder name); None for a session saved in a notebook, or shared with the dataset’s name only.

property requires: Requires#

obs columns and var genes (named under the session’s data mapping) a view shows or filters on, or a highlight group is bound to, by the key the dataset gives it. Applying another step checks that step’s own (apply()).

Type:

The dataset fields the shown state uses

property cells: dict[str, Any] | None#

The subsample the session was saved on, or None for every cell.

{"spec": {...}, "kept": numpy.ndarray, "digest": str}: the subsample’s specification (method, nObs, k, seed and the method’s own keys), the kept cells’ original positions (rows of obs, ascending, int64) and their cell-set digest. The same cells are cellucid.subsample_cells(...) of the specification.

property genes: dict[str, Any] | None#

The open cells genes were read for, or None when every open cell’s were.

{"cells": int, "order": dict | None, "digest": str}: genes were read for the first cells open cells in their order (a subsample’s priority order, or, every cell being open, the order of the blocks subsample order names); digest is the cell-set digest of their original positions.

property chrome: dict[str, Any]#

The page chrome the session carries outside its history (a copy).

{"pressed": {<group id>: <button id>}}: the pressed button of each of the web app’s pressed-button groups, which it puts back after a load and no step records, such as the selection tool ({"highlight-mode": "highlight-mode-lasso"}). {"pressed": {}} for a file written without a page. It is not part of the data: apply() ignores it.

tracker(tracker=None)[source]#

A tracker by its name or its id (None: the one the session shows).

Names are compared by their key (case and extra white space aside) among the trackers not removed; a removed tracker is named by its id. An unknown name is refused, naming the trackers there are.

Return type:

Tracker

Parameters:

tracker (str | None)

conversations(tracker=None)[source]#

The conversations kept with a tracker (its name, id, the tracker, or None: the one shown), in its order: {"conversation": id, "transcript": JSON} each. A file holds them only when the person chose to include them.

Return type:

list[dict[str, Any]]

Parameters:

tracker (str | Tracker | None)

save(dest)[source]#

Copy the session file to dest and return the destination path.

The copy is written beside dest and moved into place once complete, so an interrupted save never leaves a partial file there; saving a session onto its own file leaves it as it is.

Return type:

Path

Parameters:

dest (str | Path)

apply(adata, *, tracker=None, branch=None, step=None, dry_run=True, overwrite=False, keep_session=False)[source]#

What applying one step of this session to adata would change, or the change made.

Same as apply_cellucid_session_to_anndata() with this session; see it for the parameters, the plan’s rules and the errors.

Return type:

Any

Parameters:

  • adata (Any)

  • tracker (str | None)

  • branch (str | None)

  • step (str | None)

  • dry_run (bool)

  • overwrite (bool)

  • keep_session (bool)

The reading model (cellucid.history)#

class cellucid.history.Tracker(id, name, dataset, created_at, by, start, removed, root_id, head_id, active_branch, main_branch, paused, branches, groups, step_records, turns, notes, journal, chats, next_step, next_branch, next_group, next_note, next_journal)[source]#

Bases: Tracker

One tree of steps on the dataset, with its branches, turns, notes and journal.

Every field of the model’s Tracker (step_records holds the Step records, groups the Group records, paused None while it records, else a Pause: since when it was paused (at), who paused it and everyone who acted while it was (by), and draft, which a file keeps None); a squashed step’s squashed has carried, the notes and bookmarks of the steps it replaced (Carried); steps is the table of every step, step(id), branch(name or id) and group(name or id) one record, notes_table() every note, stale() the stale results, methods() and pseudocode() a branch in words.

Parameters:

  • id (str)

  • name (str)

  • dataset (DatasetIdentity)

  • created_at (float | None)

  • by (By)

  • start (Start)

  • removed (Removal | None)

  • root_id (str)

  • head_id (str)

  • active_branch (str)

  • main_branch (str)

  • paused (Pause | None)

  • branches (tuple[Branch, ...])

  • groups (tuple[Group, ...])

  • step_records (tuple[Step, ...])

  • turns (tuple[Turn, ...])

  • notes (tuple[Note, ...])

  • journal (tuple[JournalEntry, ...])

  • chats (tuple[tuple[str, Any], ...])

  • next_step (int)

  • next_branch (int)

  • next_group (int)

  • next_note (int)

  • next_journal (int)

property steps: DataFrame#

its id, branch (id), parent, kind (root, action, outside, squashed), action (type) and its parameters’ version (v), label, pseudocode line (code), who took it (origin, agent, model, host, turn), when (at, a UTC timestamp), whether it still applies (status), whether its result is stale at the head, its bookmark, its group (id) and its number of notes.

Type:

Every step, one row each, in step order (STEP_COLUMNS)

step(step)[source]#

One step, by its id (s12).

Return type:

Step

Parameters:

step (str)

branch(branch=None)[source]#

One branch, by its name (compared by its key) or its id; None is the active branch.

Return type:

Branch

Parameters:

branch (str | None)

group(group)[source]#

One group, by its name (compared by its key) or its id.

Return type:

Group

Parameters:

group (str)

turn(turn)[source]#

One turn, by its id.

Return type:

Turn

Parameters:

turn (str)

path_to(step)[source]#

The steps from the root to step, root first.

Return type:

tuple[Step, ...]

Parameters:

step (str)

notes_table()[source]#

Every note of the tracker, its branches, groups, steps and turns, in note order (NOTE_COLUMNS): what it is on (on: tracker, branch, group, step or turn, and target, its id), its text, its author, when it was written and last edited, and whether it was taken from an agent’s reply.

Return type:

DataFrame

stale()[source]#

The results stale at the head (history-v2 §3.4): each step on the head’s path whose output is still what the head shows while an input it read changed since, with the first later step that changed it (cause) and the inputs (domains), in step order.

Return type:

list[dict[str, Any]]

methods(branch=None)[source]#

The methods text of a branch (main when left out), as the web app writes it: the tracker’s first note opens it, and group names head its sections.

Return type:

str

Parameters:

branch (str | None)

pseudocode(branch=None)[source]#

A branch’s steps (main when left out) as pseudocode, one stored line each, groups marked with # --- QC --- comments.

Return type:

str

Parameters:

branch (str | None)

class cellucid.history.Branch(id, name, base, tip, created_at, by, lane, locked, archived, notes)[source]#

Bases: Branch

A line of steps with a unique name, from its fork step (base) to its tip.

Every field of the model’s Branch; main says whether it is its tracker’s main branch, steps is its path from the root to its tip as a table, own_steps the steps made on it.

Parameters:
property main: bool#

Whether this is the tracker’s main branch (what sharing, replay and load default to).

property path: tuple[Step, ...]#

Its steps from the root to its tip, root first.

property steps: DataFrame#

Its path from the root to its tip, one row per step (STEP_COLUMNS).

property own_steps: DataFrame#

The steps made on this branch, from its first to its tip.

class cellucid.history.Step(id, parent, children, branch, kind, type, params, v, code, label, by, at, until, merged, ms, touches, before, after, reads, status, reason, outcome, coalesce_key, replay_of, squashed, notes, bookmark, group)[source]#

Bases: Step

One step as recorded: the root, an action, a change made outside an action, or a squashed step (kind).

Every field of the model’s Step; state() is the value of every state domain after it, diff(other) what differs between it and another step, stale why its result no longer matches its inputs (or None).

Parameters:
state()[source]#

The value of every state domain after this step: {domain: value}.

A domain’s value at a step is what the nearest step on the way from the root last left it (the root holds them all). Values are plain data, as app/state/domains/README.md shapes them: typed arrays are read-only numpy arrays of the matching type (Uint32Array cells are uint32, Float32Array values float32), a number JSON cannot hold is a float (nan, inf, -0.0).

Return type:

dict[str, Any]

property stale: dict[str, Any] | None#

Why this step’s result no longer matches its inputs at the tracker’s head, or None: {"cause": step id or None, "domains": [...]}.

diff(other)[source]#

What differs between the state after this step and after other (a step of any tracker, or the id of one of this tracker’s).

Return type:

Diff

Parameters:

other (Step | str)

class cellucid.history.Diff(a, b, common, rows)[source]#

Bases: object

What differs between the states after two steps (of one tracker or two), domain by domain.

rows holds (domain, name, line): the domain’s id, its name as a person reads it, and one line of what changed, in the web app’s words for the open cells, fields and labels, gene sets and highlights, and "<name> changed" for the presentation domains. table is the rows as a DataFrame; common is the two steps’ last common step (None across trackers).

Parameters:
a: Step#
b: Step#
common: str | None#
rows: tuple[tuple[str, str, str], ...]#
property table: DataFrame#
property domains: tuple[str, ...]#

The domains that differ, in domain order.

classmethod between(a, b)[source]#
Return type:

Diff

Parameters:

apply_cellucid_session_to_anndata#

cellucid.apply_cellucid_session_to_anndata(adata, session, *, tracker=None, branch=None, step=None, dry_run=True, overwrite=False, keep_session=False)[source]#

What applying one step of a session to its AnnData would change, or the change made.

A session file holds the history of every step a person, an agent or a notebook took (CellucidSession); this applies the state one step left – its labels, user-defined fields, gene sets and highlight groups – to the object it was saved on. With dry_run=True (the default) nothing is written: the returned ApplyPlan lists every change, one row each, with what would happen to it; plan.apply() then writes it. With dry_run=False the plan is applied at once and the ApplyReport returned. Every check runs before anything is written, and everything is written at once, so a failed apply leaves the object as it was.

What is checked (only what the chosen step needs):

  • the dataset: the cell count (of the subsample the session was saved on, when it was), the data mapping’s keys and gene count, the dataset id when the object names one (uns['cellucid']['id']), and the cell order: when the object has the embedding the session names, the cell-order digest of its stored values over the session’s cells must be the session’s (the rows are those cells, in that order); a digest is a sum over the cells, so a mismatch says the cells differ, not which;

  • the fields the chosen step’s state uses (the obs columns and genes its views colour by or filter on, and its highlight groups are bound to); the session’s own requires is the shown state’s.

What a plan’s rows are, in domain order: column-renamed (an obs column renamed in place), categories-renamed and categories-merged (a category renamed, or renamed into one the column has: the two become one), field-deleted (recorded, never applied: the column stays), user-field (a column per field the session made: a categorical, or float32 values; a field deleted in the session is not applied), gene-set (uns['cellucid']['gene_sets'], and a bool var column, below), highlight (one bool column per group, cellucid_highlight__<page>__<group> from the page’s name and the group’s label, ASCII letters and digits with every other run one underscore, _2, _3… when two read alike) and presentation (a domain the step changed that is how the data was shown – filters, colours, views, the camera, analyses – never applied).

What each row’s status is: write; unchanged (the object holds it already: applying the same step again changes nothing and reports so); replace (a column or gene set an earlier apply wrote, holding other values, with overwrite=True); not-applied with the reason in note – a column or gene set the person made is never replaced.

Each gene set written or replaced also gets a bool column in the var of the genes the data mapping reads (adata.var; for a MuData, the var of the modality the mapping names), named cellucid_gene_set__<name> by the highlight columns’ rule over every set of the step (_2, _3… on a clash), True on each row whose gene, as the mapping names genes, the set lists. A new column goes after var’s columns; a replaced set’s column is replaced in place. Warned (UserWarning) when written: genes the dataset lacks (left out of the column), a var column of that name beside a set no apply wrote (the dataset’s own: kept, no column written), and a mapping that reads raw or no expression (no var to mark: the sets stay in uns only). Nothing is removed: a gene set the step no longer holds keeps its record and its column.

obs['cellucid_original_position'] (each row’s original position in a file stored in priority order) is never written, renamed or replaced: a plan that would is refused.

A session saved on a subsample, or a step taken while the viewer had a subsample open, knows the state of those cells only: a cell it did not open is <NA> in a highlight column (the nullable boolean dtype) and missing in a field’s – unknown, not “not selected”.

Parameters:

  • adata (anndata.AnnData or mudata.MuData) – The dataset the session was saved on, its cells in their original order (an AnnData read from a Cellucid-ready file in its stored priority order is refused: restore_original_order() puts it back). For a MuData the session is checked against what the viewer showed – the cells with every modality placed among them, and the modality its data mapping names – and labels and columns apply to the global mdata.obs, gene sets’ var columns to that modality’s var; mdata.uns records the apply. Written in place.

  • session (CellucidSession, str, or Path) – The session, or the path of a .cellucid-session file (read and checked whole first). One shared as its steps only cannot be applied.

  • tracker (str, optional) – A tracker by its name or its id. Default: the one the session shows.

  • branch (str, optional) – A branch by its name or its id: its tip is applied, or step when it is on the branch’s path.

  • step (str, optional) – A step by its id. Default: the branch’s tip, or with no branch the tracker’s head (what the viewer showed).

  • dry_run (bool) – Plan only (the default); False applies the plan at once.

  • overwrite (bool) – Replace a column or gene set an earlier apply wrote with other values.

  • keep_session (bool) – Keep the session file itself (base64) in the apply’s record, so the history can be rebuilt from the object alone. A record holds no source (where the viewer opened the dataset, which may be a private link); the whole file kept this way does, unless it was shared without one.

Returns:

The plan (dry_run=True), or the report of the apply: its plan, whether anything was written, and the record written to uns['cellucid']['history_applied'] (JSON text; read it with cellucid.history_applied()).

Return type:

ApplyPlan or ApplyReport

Raises:

  • TypeError – An argument of the wrong type.

  • ValueError – A session file that is damaged, of another format, written by a newer Cellucid, or shared as its steps only; a tracker, branch or step it does not hold, or a step not on the branch named; an object that is not the dataset the session was saved on, or lacks a field the step uses; a plan that would write, rename or replace obs['cellucid_original_position']; a value this version does not read.

ApplyPlan, ApplyReport and history_applied#

class cellucid.ApplyPlan[source]#

What applying one step to an object would change, written nowhere yet: the history model’s ApplyPlan (session_sha256, tracker, branch, step, requires, rows), each row an ApplyPlanRow (kind, domain, name, column, from_, cells, status, note).

property ApplyPlan.table: DataFrame#

One row per change (kind, domain, name, column, from, cells, status, note), in domain order.

property ApplyPlan.writes_anything: bool#

Whether applying it would write anything (a row to write or replace).

ApplyPlan.apply()[source]#

Write the plan into the object it was made for, all at once, and record it in uns['cellucid']['history_applied'].

The plan is made again first: an object changed since (another apply, a column added) is refused rather than half-written.

Return type:

ApplyReport

class cellucid.ApplyReport[source]#

What an apply did: its plan, whether it wrote anything (applied), and the record it wrote (provenance, the history model’s Provenance; None when nothing was written).

property ApplyReport.adata: Any#
cellucid.history_applied(adata)[source]#

Every apply recorded on an object (uns['cellucid']['history_applied']), oldest first, each the model’s Provenance as JSON.

Return type:

list[dict[str, Any]]

Parameters:

adata (Any)

Edge cases (do not skip)#

Dataset mismatch (most common)#

A session is only meaningfully applicable to the dataset it was saved on. Common mismatch causes:

  • you filtered or reordered cells before saving the session,

  • you regenerated the AnnData with different preprocessing,

  • you are applying a session to a subset.

Any mismatch raises before the target is changed.

Cell ordering assumptions#

Highlights and field values are stored by original cell position (the row in the order the dataset gives the cells). If your AnnData row order differs, labels would land on the wrong cells: when adata has the embedding the session names, the cell-order digest refuses it (a digest is a sum over the cells, so a mismatch says the cells differ, not which). An AnnData read from a Cellucid-ready file in its stored priority order is refused until cellucid.restore_original_order puts it back.

A session saved on a subsample records which cells were open (the manifest’s cells: the specification, the kept cells and their digest), and a step taken while a subsample was open records it in its cells state. Applying it checks that adata has the dataset’s full cell count; the state then applies to the cells open both when it was saved and when the step was taken. A cell that was not open is <NA> in a highlight column and missing in a field’s – unknown, not “not selected”.

A session saved while genes were read for a sample of the open cells (a large dataset stored cell by cell opens every cell on the map and reads a gene for the first of them) records that sample: CellucidSession.genes is {"cells", "order", "digest"} (how many of the open cells, the file’s cell order they were open in – every cell’s, or a share’s spread through a file in its own order – or None for a subsample’s own order, and their cell-set digest), or None. A session holds no gene values, so applying it is the same either way; the record says which cells the viewer’s gene results (a signature score, a marker table) were computed on.

Columns the object has already#

A highlight or user-defined field column whose name the object already uses is never replaced when Cellucid did not write it: its row is not-applied and says so, and the rest applies. A column an earlier apply wrote is unchanged when it holds the same values, and is replaced only with overwrite=True. Gene sets follow the same rule.

Backed AnnData#

Applying changes the object’s obs and uns in memory. A backed AnnData keeps X on disk, and its source file is not rewritten: write the object to keep the result.

Troubleshooting (symptom → diagnosis → fix)#

Symptom: “This session was saved on …, and this object has …”#

Likely causes:

  • You are applying the session to a different AnnData (another cell count, gene count, or dataset id in uns["cellucid"]["id"]).

Fix:

  • Apply to the original AnnData the session was saved on.

Symptom: “This object’s cells are not the cells the session was saved on, in the same order …”#

The cell-order digest of the session’s embedding differs: the rows are other cells, or in another order. Put the cells in their original order, or apply the session to the object it was saved on.

Symptom: “This session was written by a newer Cellucid (…) than this package reads (…)”#

The web app that saved it is newer than this package. Update the cellucid package.

Symptom: “This session was saved by an earlier version of Cellucid (session format 3) …”#

A session saved before Cellucid kept the history of every step is not read. Open it in the version of Cellucid that saved it, or re-create the session.

Symptom: “This file is not a Cellucid session file of format 4 …”#

Fix:

  • Confirm the file is actually a .cellucid-session file and not (for example) a .json file or a renamed file.

Symptom: “This session file is damaged: …”#

The file breaks the format where the sentence says (a chunk of another size than declared, a value that does not match its hash, an array no value refers to, trailing bytes…). Treat it as corrupted or tampered with, and save the session again from the viewer.

Symptom: “This session was shared as its steps only …”#

The share left the values out, so there is no state to apply. Replay it in Cellucid on its dataset, or ask for a share that keeps the values.

Symptom: “The state at step … uses the obs column … which this dataset does not have”#

The chosen step’s state colours by, filters on or binds a highlight group to fields the AnnData lacks: every missing obs column and gene is listed. Apply the session to the AnnData it was saved on (or with the same data mapping and gene names).

Symptom: “There is no step …”, “There is no branch …” or “There is no tracker …”#

The step, branch or tracker is not in the session; session.trackers, tracker.branches and tracker.steps list what it holds.

Symptom: a row is not-applied#

Its note says why: a column or gene set the person made, a column or category the object does not have (a column dropped since, a modality’s column of a MuData), a deletion, a presentation domain. Everything else was applied; cellucid.history_applied(adata) lists what each apply skipped.

Symptom: “The object changed since this plan was made …”#

Something changed adata between the dry run and plan.apply(). Make the plan again and apply that.

Symptom: “No highlight columns were added”#

Likely causes:

  • The state at that step holds no highlight group.

How to confirm:

  • tracker.step(step).state()["highlights"]["pages"].

See also#

On this page