App architecture overview#

This page is the big-picture mental model for the Cellucid web app: how the app boots, how data moves through the system, and where performance-sensitive boundaries are.

If you are new to the codebase, start here before diving into any single module.

At a glance#

Audience

  • Wet lab / non-technical: read “Big picture” + “What is state?” + “How to report a bug”.

  • Computational users: read “Boot sequence” + “Data sources” + “State and persistence”.

  • Developers: read the whole page, then continue to State (DataState) and events.

Time

  • 20–40 minutes

Prerequisites

  • None (links point to code files in this workspace)

Big picture (boot → state → UI → rendering)#

cellucid/index.html
  ├─ defines: sidebar DOM + <canvas id="glcanvas">, CSP, deferred (disabled) controls
  ├─ loads:   ui/core/*-init.js, code-load-watch.js, file-drop-guard.js (classic, pre-module)
  └─ runs:    assets/js/app/main.js  (ES module)

main.js (orchestrator)
  ├─ createViewer()     → rendering/viewer.js (WebGL2 renderer)
  ├─ createDataState()  → app/state/core/data-state.js (typed arrays + state managers)
  ├─ initUI()           → app/ui/core/ui-coordinator.js (DOM + modules)
  ├─ data sources       → data/data-source-manager.js (sample catalog, files, server, links, viewer)
  ├─ data loaders       → data/data-loaders.js (AnnData adapters over the reader worker)
  ├─ sessions           → app/session/session-serializer.js (.cellucid-session, format 4)
  ├─ histories          → app/history/dataset-history.js (a history per dataset shown)
  ├─ state domains      → app/state/domains/index.js (installStateDomains)
  ├─ analysis           → app/analysis/* (compute backends + plots + UI)
  └─ admits deferred controls once their listeners exist

DataState (state coordinator)
  ├─ emits events: visibility/field/highlight/page/view-representation/mapping/vectorFields changes
  ├─ owns: typed arrays + per-view contexts
  └─ calls viewer: updateColors/updateTransparency/updatePositions/etc

  the one-way rule
  ───────────────
  control ──dispatch──▶ action ──handler──▶ DataState ──explicit method──▶ viewer ──▶ GPU
     ▲                    │ (app/actions/)       │
     │                    └──recorded──▶ history │ (app/history/)
     └────────────────event──────────────────────┘
  A control dispatches the action it stands for and calls no mutating state
  method; a UI module never reaches past DataState into the viewer, and the
  viewer never reads the DOM. These shortcuts have been the source of
  regressions.

Design goal: keep “hot” work (per-frame render and per-point math) inside the renderer and state managers, not in main.js or UI modules.

Boot sequence (what happens on page load)#

The high-level startup flow is:

  1. cellucid/index.html declares its import map (every module at the URL naming its bytes), runs its classic bootstraps (ui/core/preconnect-init.js, ga-init.js, theme-init.js, viewport-init.js, code-load-watch.js, file-drop-guard.js), then loads assets/js/app/main.js as an ES module. The parser stops at the style gate (<script id="cellucid-style-gate">, CELLUCID_STYLE_GATE, straight after the stylesheet link) until the stylesheet and all it imports are in, so no engine parses <body> or fires DOMContentLoaded, the boot’s cue, in a page without its styles (WebKit does not hold DOMContentLoaded for a stylesheet by itself). main.js marks the root data-code-started once its whole static graph has run; an uncaught error before that mark, a module that did not download, or a stylesheet that did not, is the code-load watch’s Cellucid could not load card.

  2. main.js creates a WebGL2 viewer: createViewer({ canvas, labelLayer, viewTitleLayer }). The full signature also accepts onViewFocus; the bootstrap does not pass it. If the canvas yields no webgl2 context this throws immediately and the startup-failure card replaces the app. There is no software renderer.

  3. main.js creates the app state: createDataState({ viewer, labelLayer }).

  4. main.js initializes notifications and the optional analytics entry point (app/optional-analytics.js): it queues events while the analytics modules load by dynamic import, is never awaited, never throws, and stays inert when a privacy blocker refuses them. Then it sets up the DataSourceManager.

  5. The app decides what to load first — it resolves the startup selection (source, dataset id, load method) but loads nothing yet:

    • URL params (?remote=…, ?data=gh:…, ?dataset=…) may override the default demo.

    • A viewer page a host program opened (?viewer=true&viewerId=…&viewerToken=…, Python show_anndata() or R cellucid_view()) registers the jupyter data source and selects the dataset its server serves.

  6. The app publishes the empty runtime and initializes the UI coordinator (initUI), which wires the sidebar modules to state/viewer.

  7. Only once their listeners exist are the deferred controls (data sources, renderer toggles, benchmark) enabled, and the live-region line under the Open data heading (#data-source-status) switches from “Starting Cellucid…” to “Data sources are ready…”. The render loop starts.

  8. The startup selection loads as the first ordinary dataset change (ui.loadStartupDataset → reloadActiveDatasetInPlace), under the same transaction owner as every later change, so any choice the user makes while it loads replaces it:

    • the AnnData file opened in the reader worker, then the identity, obs and var inventories and the graph description the AnnData adapter builds from it

    • points buffers (embedding positions, colors, outlier quantiles, etc.)

    • the dataset’s history, made once the dataset is published; the starting view its catalog names, when it names one, put back without a step; then the first tracker starts. A dataset the person chose (a link, a file, a sample picked; not the startup’s default sample) is recorded in My datasets A failure of this load that nothing superseded is the terminal startup failure; otherwise startup publishes ready when it settles.

Important

Step 7 is an invariant, not a nicety. Any control whose handler is wired after an await in this bootstrap must be authored disabled in index.html and registered with ui/core/deferred-control-readiness.js. A control that is clickable before its listener exists silently discards the click, which is indistinguishable from a bug to the user.

Data sources (how the frontend “gets data”)#

Cellucid supports multiple data sources through a single coordinator:

  • cellucid/assets/js/data/data-source-manager.js (DataSourceManager)

    • Registers sources and manages dataset switching.

    • Produces a dataset base URL (baseUrl) used by the low-level loaders.

Common sources:

  • local-demo: the sample dataset list, read from a dataset catalog (datasets.json and the AnnData files it lists) at the sample catalog URL (configured via <meta name="cellucid-sample-catalog" ...> or ?sampleCatalog=...; in production https://theislab.github.io/cellucid-datasets/anndata/)

  • local-user: Open… → File… or Folder… (and drag and drop) for one file or folder of any format in data/input-formats.js (AnnData .h5ad and Zarr, MuData, Loom, 10x .h5 and outs/, R objects, MatrixMarket folders)

  • remote: connects to a cellucid serve server; the browser’s AnnData reader reads the file or store it names in place with range requests

  • link: data opened from a link through a provider of data/sources/registry.js (GitHub, CELLxGENE, GEO, Zenodo, Hugging Face, Figshare, Dropbox, Google Drive, OneDrive) or any web address of a file, a store folder or a dataset catalog folder (sharing; no server)

  • jupyter: the viewer a host program opened (Python notebook or script, R), with its command/event channel (data/viewer-channel.js)

Deep dive: Data loading pipeline and caching.

What is “state” in Cellucid?#

Cellucid has many interacting features. The app treats state as first-class:

DataState is the core coordinator#

DataState (in cellucid/assets/js/app/state/core/data-state.js) owns:

  • core typed arrays (positions, colors, visibility/transparency)

  • field registries (rename/delete/user-defined)

  • highlight pages/groups and per-point highlight overlay arrays

  • per-view contexts (live view + snapshot views)

  • the representation (embedding) each view shows, the data mapping, and vector-field availability

  • loaded obs/var field data, released least recently used once inactive fields pass their memory budget (field/residency.js, data/memory-budget.js)

DataState emits events so UI/analysis modules can stay decoupled:

  • visibility:changed

  • field:changed

  • highlight:changed

  • page:changed

  • view:representation-changed

  • mapping:changed / representations:changed

  • vectorFields:changed

Deep dive: State (DataState) and events.

Viewer state is separate#

The WebGL viewer (cellucid/assets/js/rendering/viewer.js) owns GPU resources and the render loop. It is updated by the state layer using explicit methods like:

  • viewer.setData(...)

  • viewer.updateColors(...)

  • viewer.updateTransparency(...)

  • viewer.updatePositions(...)

Deep dive: Rendering pipeline (WebGL) and performance notes.

UI architecture (how the sidebar is wired)#

The UI is intentionally modular:

  • cellucid/assets/js/app/ui/core/dom-cache.js collects DOM element references once.

  • cellucid/assets/js/app/ui/core/ui-coordinator.js initializes modules and wires callbacks.

  • cellucid/assets/js/app/ui/modules/* contains feature modules (filters, highlights, sessions, export, etc.).

Deep dive: UI modules map.

Actions, state domains and the history#

Every state change a person, an agent, a Python or R host or a replay can make is an action of one catalogue (app/actions/), dispatched through one dispatcher that checks its parameters, orders it against the actions running on the same state, records it as one step of the history and rolls a failed one back. A control dispatches the action it stands for (dispatchFromControl); settings controls and camera gestures are recorded by observation. The state is split into fifteen state domains (cells, mapping, field-overlays, … layout), each with one owner in app/state/domains/ that captures and restores its value through the DataState methods the controls use. The history (app/history/) keeps trackers of step trees and every domain value by content hash, so undo, redo and a jump put values back instead of recomputing. Deep dive: Action layer, state domains and history.

An agent (the chat bar with the person’s API key, Claude or ChatGPT through the Cellucid relay, or a local app through cellucid mcp) acts through the same catalogue: each action is one tool, every call runs through one tool call (app/agent/bridge/tool-call.js) as origin agent, the person’s permissions decide whether it runs, asks or is refused (app/agent/presence/), and its steps undo as one turn. Deep dive: Agents in the tab: chat bar, permissions, data guard and relay.

Persistence and reproducibility#

Cellucid has multiple persistence-like mechanisms; they are different and must not be conflated:

  1. Session file (.cellucid-session, session format 4)

    • A downloadable, shareable file holding the dataset’s history: its trackers, their steps and every state value they refer to (or, as chosen in State › Session › Share…, the current state only, or the steps only as a replay plan). Load… adds its trackers beside the person’s and shows the active step.

    • Code: cellucid/assets/js/app/session/ (the file) over cellucid/assets/js/app/history/ (what it holds). cellucid/assets/js/app/state-serializer/ is only the sidebar’s control inventory (ui-controls.js), which the settings and layout domains read.

  2. History kept in this browser (the autosave and named saves)

    • Every step written to IndexedDB cellucid-history as it is made; reopening the dataset offers to continue (app/history/autosave.js).

  3. My datasets

    • The datasets opened in this browser and how each opens again, in IndexedDB cellucid-library (app/library/my-datasets.js); exported and imported with their saved states as one .cellucid-library file (app/library/library-file.js).

  4. Browser storage (preferences/caches)

  5. URL state (deep links)

Deep dive: Sessions: persistence and serialization.

Performance invariants (rules contributors should treat as “hard”)#

These are the most common sources of accidental regressions:

  • No per-frame DOM work: the render loop should not query or update DOM every frame.

  • No hot-path allocations: avoid allocating arrays/objects in per-point loops; reuse typed-array scratch buffers.

  • State → viewer updates should be explicit: prefer viewer.update* methods rather than “implicit” coupling.

  • Batch expensive recomputations: when restoring filters or applying many changes, bracket the mutation with DataState.beginBatch() and DataState.endBatch().

  • A value captured for the history is never written again: a domain owner copies what the app writes in place, and caches by identity what it replaces, so capturing an unchanged domain costs nothing per cell.

Cancellation and failures (one rule each)#

Every thread of the app (the page, the reader, decode, analysis and figure-export workers) follows the same three rules. A contract test (tests/cancellation-guard-contract.test.mjs) keeps every module on them.

  • One way to stop work (assets/js/utils/abort.js). Work stops through cancel(controller, kind, words) or a controller aborted with cancelled(kind, words). A cancellation is an AbortError with a kind: user (the person pressed Cancel or closed something), superseded (a newer choice replaced it), closed (what it belonged to went away), released (memory was given back under it; asked again, it works) or page-left. isCancellation(error) answers by name alone, so the platform’s own aborts count too, and cancellationKind(error) tells the kinds apart. A cancellation is never shown as a failure. Shared work (SharedWork) stops only once every waiter has left, and its own abort names no waiter’s reason. awaitWithSignal and onAbort keep one listener on a signal however many reads wait on it. They are the only way to wait on a signal: nothing calls addEventListener('abort', …) itself (tests/abort-listener-contract.test.mjs). An abort event fires once and is not replayed, so a listener added to a signal that has already aborted never runs. That is how a text-matrix pass stopped before it began once read the whole matrix. onAbort calls back at once when the signal has already aborted, before it returns. It returns a remover that the site calls when its work settles, so a dataset-long signal keeps no callback of finished requests. Work started with a stopped signal does nothing (tests/abort-before-start-contract.test.mjs). No module defines its own abort helpers, aborts with a plain Error, builds an AbortError by hand, checks name === 'AbortError', or uses a subsystem code for supersession.

  • No clock turns slowness into failure. An analysis task, a probe of a Cellucid server and a request to a viewer host have no deadline of their own. What bounds a wait is its owner’s signal (Cancel, a newer choice, closing) or measured silence (startup steps, an opening’s stall note). A request to a notebook kernel counts as a wait on the person (utils/person-wait.js): the kernel answers between cells. A kernel that goes away is said, not timed (data/viewer-embed.js, ended).

  • One wire format, one notice (utils/error-wire.js, utils/errors.js). Every worker answers a failed request with toWire(error) and the page rebuilds it with fromWire, so a refused allocation stays a RangeError, a cancellation keeps its kind, and an explained failure keeps its words, code and cause. The surface that shows a failure marks it (markReported), and a caller further up skips a marked one (isReported): a gene that fails to load is said once, by its loading card. Background work nobody awaits (a kept journal’s removal) is observed with settleQuietly(promise, doing).

A worker that never loaded (a page that outlived its deploy) is retired, and its calls fail saying “reload the page”. A stray error in a live worker is logged and fails none of the reads waiting on it. A page put in the back/forward cache keeps its reader worker, so the person comes back to reads still under way. Cellucid checks before startup that the browser has the newest platform features it calls (BROWSER_FEATURES in app/startup-failure.js: AbortSignal.any, which sets the floor at Chrome 116, Firefox 124 and Safari 17.4).

Troubleshooting: how to report a bug in a way that can be fixed#

If you are reporting an issue (even as a non-developer), include:

  1. Environment

    • Browser + OS

    • Hosted (https://www.cellucid.com) vs local app (http://localhost:8000) vs a Python/R viewer (notebook or browser tab)

  2. Dataset identity

    • dataset id (and where it came from: local-user folder, remote server, GitHub repo/path)

  3. Exact steps

    • “click X, then select Y, then move slider Z”

  4. Expected vs actual

    • What did you think would happen?

    • What happened instead?

  5. Console + network clues

    • Turn on debug: localStorage.setItem('CELLUCID_DEBUG','true'); location.reload();

    • Copy the first error stack trace and any failed network requests

Deep playbook: Debugging playbook.

Next: State (DataState) and events.