Community Annotation — Author Guide (Repo Setup + Operations)#
This guide is for dataset authors / maintainers who want to run a community annotation round and keep it healthy at scale.
If you are an annotator (voting only), go to Community Annotation — Annotator Guide (UI + Voting).
How To Read This Guide (Two Tracks)#
Follow these sections in order:
Section 0 (checklists): avoid the common “we started too early” problems.
Section 2 (dataset id): confirm your dataset id is stable.
Section 3–6 (repo + app): create repo, configure
config.json, install GitHub App.Section 7–9 (Cellucid UI): connect, enable fields, tune consensus settings.
Section 12 (export): download
cellucid-consensus.json.Section 13 (troubleshooting): when something breaks.
Read everything, but pay special attention to:
Section 0.1 (people/process): who merges PRs, who merges duplicates, and how decisions are communicated.
Section 4 (what a file may hold, and what a Pull leaves out).
Section 5 (GitHub settings): fork/PR flow, CI validation, and branch strategy.
Section 9 (consensus parameters): how thresholds behave under disagreement.
Section 10 (merges): when to merge vs when to leave disputed.
Section 13 (rate limits, storage, and “it works for me” failures).
Limits: what GitHub and Cloudflare bound, and what Cellucid cannot check.
Starting point in the sidebar#
Before repository setup, the Community Annotation panel reports that no repository is connected and presents the explicit Connect repo action.#
2) Choose a Stable Dataset ID (Critical)#
Community annotation is scoped by dataset id. If the id changes, existing annotations will not appear (it becomes a different scope).
How dataset id is determined#
For a file that states its id in
uns['cellucid']['id'], Cellucid uses that id, whatever opened it: the sample menu, a data link, a server, a notebook, or Open… for a local file or folder. The same file keeps the same annotations everywhere.A file without one gets an id from how it was opened: a data link gives a stable one, such as
cellxgene:<dataset id>orgeo:<GSE>:<file>(Open from a link); a local file gets one from its format, its file name, its cell and gene counts and its gene names (h5ad_atlas_1bd42f4e). Writing the same data again (an observation column added) keeps that id; two files with the same name, cell and gene counts and gene list share annotations unless they carry their own id, whichcellucid.prepare()andcellucid_prepare()write. See Dataset identity (why it matters).When writing the file with
cellucid.prepare(...), setdataset_id=....
One repository can annotate several datasets. Each contributor’s file keeps a
separate section per dataset, so two datasets that share a field and a
category label (for example leiden / 0) never share suggestions, votes or
merges.
Example (recommended):
from cellucid import prepare
prepare(
"my_atlas.zarr",
format="zarr",
embeddings={"X_umap": adata.obsm["X_umap"]},
obs=adata.obs,
dataset_id="my_atlas_v1", # keep this stable for the entire annotation round
dataset_name="My Atlas (v1)",
)
How to confirm:
import anndata as ad
ad.read_zarr("my_atlas.zarr").uns["cellucid"]["id"] # 'my_atlas_v1'
Tip
Treat dataset id like a contract:
If you write the file again with small technical changes but the same clusters and meaning, keep the same id.
If you change clustering, category labels, or biological meaning, use a new id and create a new round (or a new
supportedDatasets[]entry).
Confirm the dataset id in the UI (recommended)#
In Cellucid, the Community annotation panel’s status block displays the dataset id (this is the id your annotations/config.json must match).
The status block at the top of the connected panel. Dataset is the id your
annotations/config.json must carry verbatim; GitHub and Repo complete
the scope your local annotation state is filed under.#
3) Create the Annotation Repository#
Cellucid expects a specific repository layout. The easiest path is to start from the template and customize it.
Required layout (paths are case-sensitive)#
annotations/
config.json
config.schema.json
schema.json
users/
.gitkeep (one LF byte; keeps the empty folder in Git)
(one JSON file per contributor, ghid_<GitHub user id>.json)
moderation/
merges.json (optional but recommended; authors publish it from the UI)
merges.schema.json
.github/
workflows/
validate.yml (recommended)
scripts/
validate_user_files.py
.gitattributes
The browser requires both annotations/config.schema.json and
annotations/moderation/merges.schema.json even though the active
annotations/moderation/merges.json document itself is optional.
The three schema files must be the template’s, byte for byte. Cellucid
identifies them by content: it reads each schema’s Git blob id from the
repository and accepts the repository only when all three are exactly the
files published in
theislab/cellucid-annotation
(user files version 3, merges version 3, config version 1). It never downloads
them. A schema that differs in any byte (an older version, an edit, other line
endings) stops Pull with a message that names the file and asks you to copy it
again. .gitattributes keeps every file’s line endings LF on every platform,
so a checkout on Windows commits the same bytes; copy it with the schemas.
Recommended “template-first” setup#
The template repository is theislab/cellucid-annotation. Copy only its layout files, at the same paths:
annotations/(withannotations/users/.gitkeep; replace the exampleannotations/config.jsonentry with your own)scripts/validate_user_files.py.github/workflows/validate.yml.gitattributes
Leave the template’s own tests/, scripts/generate_contract_corpus.py,
.github/workflows/tests.yml, licence and policy files behind: they test and
govern the template, not your annotation repository.
Create a new GitHub repository (public or private).
Add the layout files listed above at the same paths, with their content unchanged (the schemas byte for byte).
Update
annotations/config.json(Section 4).Commit + push.
If you’re doing this in the GitHub web UI:
create the folders/files exactly as shown,
copy/paste file contents carefully (JSON is strict: no trailing commas); for the three schema files, upload the files themselves rather than pasting them, so not a byte changes.
Create an empty repo on GitHub.
Clone it locally.
Copy the layout files into the repo root.
Edit
annotations/config.json.Commit + push.
Example on macOS/Linux (from a clone of cellucid-annotation):
cd /path/to/cellucid-annotation
rsync -aR annotations scripts/validate_user_files.py .github/workflows/validate.yml .gitattributes /path/to/your-annotation-repo/
cd /path/to/your-annotation-repo
git add -A
git commit -m "Initialize Cellucid annotation repo"
git push
rsync -R keeps each path relative to the template root, so the files land
at the same paths in your repository.
Windows PowerShell equivalent:
Set-Location 'C:\path\to\cellucid-annotation'
$dest = 'C:\path\to\your-annotation-repo'
foreach ($p in 'annotations', 'scripts\validate_user_files.py', '.github\workflows\validate.yml', '.gitattributes') {
$parent = Split-Path (Join-Path $dest $p)
New-Item -ItemType Directory -Force -Path $parent | Out-Null
Copy-Item -Recurse -Force $p $parent
}
Set-Location $dest
git add -A
git commit -m "Initialize Cellucid annotation repo"
git push
Note
Keep this repo “boring”:
Avoid large binaries (screenshots belong in documentation repos, not the annotation repo).
Avoid data exports (the annotation repo should contain only JSON + scripts).
Avoid rewriting history (force pushes) once annotation starts; it confuses caches and PR history.
Branch strategy (do not skip)#
Everyone must be on the same owner/repo@branch, or they will appear to “disagree” because they are literally writing to different universes.
Common strategies:
Simple: use
mainfor the whole round.Safer: create a dedicated branch for the round (e.g.
round-2025-01) and tell everyone to use it.
For large groups, a dedicated branch reduces accidental changes to main and makes it easier to archive rounds.
4) Configure annotations/config.json#
annotations/config.json binds your annotation repo to one or more dataset ids and specifies which columns are annotatable.
Minimal example (one dataset, one field)#
{
"version": 1,
"supportedDatasets": [
{
"datasetId": "my_atlas_v1",
"name": "My Atlas (v1)",
"fieldsToAnnotate": ["leiden"],
"annotatableSettings": {
"leiden": { "minAnnotators": 3, "threshold": 0.5 }
},
"closedFields": []
}
]
}
What each field means (and what can go wrong)#
version(must be1)If you change this, validation will fail.
supportedDatasets(must be a non-empty array)Each entry is one dataset you want this repo to serve.
Dataset ids must be unique.
supportedDatasets[].datasetIdMust match the dataset id of the dataset currently open in Cellucid: the id its file states in
uns['cellucid']['id'], or the one its route gives it (Dataset identity (why it matters)). When the repo does not list it, Cellucid names the id it looked for.If the currently loaded dataset id is missing:
annotators are blocked (cannot Pull / view annotations)
authors can still connect (with a confirmation) and Publish an updated config to unblock everyone
supportedDatasets[].nameHuman-friendly name shown in the UI.
fieldsToAnnotateList of categorical obs keys (column names) that may be annotated.
Every listed key must exist as a categorical field in the loaded dataset. Any missing key fails the complete Pull without changing annotation state. Cellucid performs this check before selecting the raw-cache scope or downloading any user or moderation files, so the mismatch cannot mutate the raw-file cache.
annotatableSettings[fieldKey]Per-field consensus rules.
minAnnotators(whole number, 0 or more; the only upper bound is 9,007,199,254,740,991, the largest integer a browser holds exactly): minimum unique voters required before a bucket can be anything other than “Pending”. A bucket nobody voted on is always “Pending”, even withminAnnotators: 0.threshold(number -1..1): minimumconfidenceto reach “Consensus”.
closedFieldsFields in this list are locked for annotators (no voting/suggestions/comments).
Validation rule: every closed field must also be in
fieldsToAnnotate.
Important
Validation rule (enforced by the template CI script):
The keys in
annotatableSettingsmust be exactly the same keys asfieldsToAnnotate; neither side may contain an extra or missing key.Every key in
closedFieldsmust also appear infieldsToAnnotate.
If you violate this, GitHub Actions will fail and authors may be blocked from publishing updates cleanly.
How threshold behaves (do not guess)#
Cellucid computes (per category bucket):
voters: unique users who cast any vote in that bucket (across all suggestions)netVotes: for the current leading suggestion,upvotes - downvotesconfidence = netVotes / voters(ranges from-1to+1)
Important edge cases:
If
voters < minAnnotators→ status is Pending (even if there is a strong early leader).If nobody voted (
voters = 0) → status is Pending, whateverminAnnotatorsandthresholdsay.Suggestions with exactly the same label count as one suggestion (Section 10);
voterscounts each person once.Suggestions are ranked by net vote first and upvote count second. A top tie exists only when suggestions have both equal net vote and equal upvote count; that tie is Disputed.
See Section 9 for worked examples and recommended defaults.
Validate your repo inputs (recommended)#
In the annotation repo (not in Cellucid), run:
python scripts/validate_user_files.py
This validates (Python 3.10 or newer; it reports every problem as an error, never as a crash):
that the three schemas are the canonical ones, byte for byte
annotations/config.jsonevery entry under
annotations/users/: onlyghid_<id>.jsonuser files and the.gitkeepsentinel may be there (no other file, folder or link)annotations/moderation/merges.json(optional)that no path is a case variant of a layout path (such as
annotations/Users/), which Cellucid never reads
If this fails, fix the file(s) it reports before inviting annotators.
What a file may hold#
No text has a length limit: labels, evidence, ontology ids, marker genes, comments, merge notes, profile fields, dataset ids and names and field keys can be as long as the file allows. Every text is exact: its first and last characters are not whitespace, and it is never trimmed. Whitespace is one character class everywhere (the schemas, the validator and the app), the union of JavaScript’s
\sand Python’sstr.isspace().The only size bound is the file: every active annotation document (
annotations/config.json, eachannotations/users/ghid_<id>.json, andannotations/moderation/merges.json) is at most 1,000,000 UTF-8 bytes, the most GitHub’s repository Contents API returns as JSON (GitHub: get repository content). One contributor’s file holds every dataset they annotated in this repository, so this bound covers all of them together.Integers are at most 9,007,199,254,740,991.
JSON nesting is at most 512 levels in both validators (Python’s JSON reader recurses once per level and stops at the interpreter’s recursion limit, 1000 by default); no valid annotation file nests deeper than 7.
Suggestion ids name their owner:
ghid_<id>_<rest>, whereghid_<id>is the proposer’s own identity (proposedBy) and<rest>is exact text without:(Cellucid writesghid_<id>_<random UUID>). Because the digits end at that_, no contributor can write an id that collides with, or claims to be, another contributor’s suggestion. Votes, comments and merges may name anyone’s suggestion; a deletion marker (deletedSuggestions) names only the file owner’s own.
Pull: what stops it and what is left out#
A Pull has no limit on how many user files there are or on their total size (GitHub’s own listing limit is under Limits below). It reads the repository at one commit, downloads only the files whose Git blob id changed since the last completed Pull, keeps each batch in the browser’s file cache as it arrives (an interrupted Pull resumes there), and compiles the merged view by streaming the cache one file at a time.
One file never stops a Pull. A user file or merges file that is not
valid (malformed JSON, over 1,000,000 bytes, failing its schema, a version
other than 3) is left out of the consensus on its own: Cellucid keeps the
reason with the file’s blob id (so it is not downloaded again until it
changes), shows one warning that says how many files were left out, and lists
each file with its reason under the warning’s details. For the open dataset,
Cellucid reads a user file’s profile and that dataset’s section only, so a
broken section of another dataset does not leave the file out here. A path
under annotations/users/ that is not a ghid_<id>.json file, and a case
variant of a layout path (annotations/Users/), is ignored and listed the
same way. If your own published file is the one left out, the warning says so
first: Publish will not overwrite it until you fix or remove it on GitHub.
These stop a Pull, because nothing can be compiled without them:
a schema file that is not the canonical one, a missing
annotations/config.jsonorannotations/users/folder, or a missing branch;an invalid
annotations/config.json;a listing of
annotations/users/that GitHub reports as truncated;a configured categorical field the open dataset does not have. That check fails the complete Pull without changing annotation state, and it runs before selecting the raw-cache scope or downloading any user or moderation files, so the mismatch cannot change the file cache.
A Pull that stops leaves the annotation view as it was.
5) Configure GitHub Repo Settings (Highly Recommended)#
Decide how annotators will publish#
You have two viable models:
Direct publish (annotators have write access)
Pros: simplest experience (Publish writes directly to
annotations/users/ghid_<id>.json)Cons: requires adding many people as collaborators; less review control. Anyone with push access can also write any file with Git or GitHub’s web editor, including another contributor’s
ghid_<id>.json: Cellucid’s Worker refuses such a write made through Cellucid, but Cellucid reads files by content and cannot tell who committed them. Give push access only to people you trust with every file, and review the commit history when in doubt.
Fork + Pull Request publish (annotators do not have write access)
Pros: reviewable contributions, no direct writes to your repo
Cons: you must merge PRs; depends on forking being allowed and not blocked by org policy
Cellucid chooses the best option per user:
If the user can push → direct publish
Else if the repo allows forking → fork + PR publish
Else → user cannot publish (they can still vote locally, but nothing can be shared)
In other words, Cellucid uses a fork + Pull Request only when the repository allows forking. It selects exactly one route from repository metadata before mutation; a route failure is terminal and does not switch to the other route.
Fork + PR model: what a contributor does first#
Cellucid never creates a fork. A contributor without write access does two things on GitHub before their first Publish:
Fork the repository:
https://github.com/<owner>/<repo>/fork(any name; Cellucid finds a renamed fork among the repository’s forks).Install the Cellucid GitHub App on their own account with access to the fork: All repositories, or Only select repositories with the fork selected (a fork made after the installation is added under Settings → Applications → Installed GitHub Apps → Configure). A user token reaches a repository only in an account where the app is installed (GitHub: authenticating on behalf of a user), and the head branch and the user file are written in the fork. Opening the pull request uses the app’s installation on your repository.
What Publish says otherwise:
No fork:
You have no fork of <owner>/<repo> yet: fork it on GitHub, then Publish again., with a Fork on GitHub link tohttps://github.com/<owner>/<repo>/fork(also under Details). Nothing is written.A fork GitHub is still copying (GitHub creates forks asynchronously):
GitHub is still copying your fork <login>/<repo>; Publish again in a minute.Cellucid asks once and does not wait.The app cannot write to the fork:
Cellucid cannot write to your fork <login>/<repo>: install Cellucid's GitHub App on your account with access to it, then Publish again., with an Install the app link.
How a fork publish works, so the pull requests you receive make sense:
Each contributor has one pull request per base branch, from one head branch in their fork named
cellucid-annotations/ghid_<id>/…. It carries their whole user file, so every dataset they annotate in this repository goes into the same pull request.On every Publish, Cellucid starts from the file in that open pull request (it holds sections not merged yet) or, without one, from the file on your branch, replaces the open dataset’s section, then resets the head branch to your branch’s current tip and writes the file on it. The pull request’s diff is that one user file, and a pull request you squash-merged earlier never makes the next one conflict. An open pull request is reused; a new one is opened only when none is open.
Enable validation CI#
The template includes:
scripts/validate_user_files.py.github/workflows/validate.yml
The shipped validate.yml runs one command, python scripts/validate_user_files.py,
on every push and pull request, with read-only repository access
(permissions: contents: read). It needs nothing but the layout files, so it
works in a repository that copied only those.
Suggested GitHub settings:
Require the validation check to pass before merging PRs.
For fork-based contributions, allow GitHub Actions to run on PRs (org policies may apply).
CI is a gate for pull requests and a report for everything else: a direct publish lands before CI runs. A file CI would reject is left out on Pull with its reason (Section 4), never silently used.
Branch protection (advanced, but important)#
Branch protection can break direct publishing:
If direct pushes are blocked, users with “write” permissions may still see Publish fail.
There is no UI route override. Branch protection alone does not select fork + Pull Request: when repository metadata reports push permission, Cellucid selects direct publication and reports a terminal error if GitHub rejects it.
For direct publication, configure the connected branch to accept the exact Cellucid commit. For an annotator to use fork + PR instead, that annotator must have no source-repository push permission and the repository must allow forking. Authors with maintain/admin permission remain on the direct route; use GitHub outside Cellucid for an all-PR governance policy that blocks their direct commits.
6) Install the GitHub App (Required for Repo Discovery)#
Cellucid’s UI lists only repositories where the Cellucid GitHub App is installed.
Install the app on the user/org that owns the annotation repo.
If you choose “Only select repositories”, make sure the annotation repo is selected.
Note
Org repos often require an org admin to approve the installation.
Optional: self-host the GitHub OAuth + API proxy (org deployments)#
Cellucid’s community annotation UI uses a small server component (the checked-in deployment is a Cloudflare Worker) to run GitHub App OAuth, proxy only the repository operations used by Cellucid, run two GraphQL reads it writes itself, and relay fixed CAP persisted queries.
If you are using cellucid.com, you typically do not need to do anything here.
If your organization requires owning the auth infrastructure (recommended for many orgs), you can self-host.
Exact version-3 route contract#
The Worker exposes the following ordered routes:
/auth/login/auth/callback/auth/revoke/auth/user/auth/installations/auth/installation-repos/cap/lookup-cells/graphql/snapshot/graphql/blobs/api/repos/*
Opening / must return the exact compatible service identity and ordered route
inventory, beginning
{"status":"ok","service":"Cellucid GitHub Auth","contractVersion":3,
and continuing with an endpoints array containing the routes above. Before
OAuth or token disclosure, the browser checks that health document and stops
on a stale or incompatible Worker. An older Worker is reported as “The
community-annotation service at <origin> runs an older version (contract N)
than this Cellucid (contract 3); its owner must redeploy the Worker.”; a newer
one asks you to reload Cellucid. A Worker that cannot be reached at all is
reported as unreachable, not as incompatible.
What the routes do, in short (the setup guide has the exact contract):
/graphql/snapshotreads a branch’s commit and the listings ofannotations/andannotations/moderation/in one GraphQL query the Worker writes itself;/graphql/blobsreads many files by blob id in one query. The browser never sends a GraphQL document./auth/installationsand/auth/installation-reposanswer one GitHub page (100 items) per request (?page=N), so a list of any length is read page by page./api/repos/*relays the repository requests Cellucid uses, among them a users folder read by its tree id and the forced move of a contributor’s own pull-request head branch (PATCH git/refs/heads/cellucid-annotations/ghid_<id>/…in the contributor’s own fork).Every write is bound to the signed-in user, whom the Worker looks up at GitHub first: a user file
annotations/users/ghid_<n>.jsononly by usern, a branch undercellucid-annotations/only as that user’s owncellucid-annotations/ghid_<id>/…(created or moved only in a repository the user owns), a pull request only from such a branch. A refused write is403with outcomenot-applied.Every answer that relays GitHub carries GitHub’s
ETag,Retry-AfterandX-RateLimit-*headers, so the browser paces its requests by GitHub’s budget; reads acceptIf-None-Match, and a304costs no transfer. Every route caches a CORS preflight for 7200 seconds (Access-Control-Max-Age: 7200, the longest Chromium keeps one).A request has no total deadline: it is stopped (
504) only when nothing moved for 13 seconds (GitHub’s own 10-second request timeout plus a connection set-up), so a slow but steady upload or download is never cut.
Exact Worker and GitHub App configuration#
The Worker requires exactly ALLOWED_ORIGINS, GITHUB_CLIENT_ID, and
GITHUB_CLIENT_SECRET. It does not use an App id, private key, App JWT, or
installation access token. It deploys on the Cloudflare Workers Free plan
(about 10 ms of CPU and 50 subrequests per request). The checked-in Worker
stays well inside both at any repository size: it relays file contents, trees
and GraphQL answers without parsing them, checks a write with native JSON
parsing, and needs at most two GitHub subrequests per request. A request that
did exceed the CPU budget would be answered by Cloudflare with an HTML 503
page without CORS headers, which the browser can only report as an unreachable
Worker; the setup guide explains how to diagnose that and when to move to
Workers Paid.
On the Free plan, abuse of the Worker cannot create a bill: it can only use up
the account’s daily request allowance (100,000 requests, reset at midnight
UTC), after which Cloudflare answers with an error until the next day.
Cloudflare’s rate-limiting rules belong to a zone (a domain you have on
Cloudflare), so they cover a custom domain and never a *.workers.dev
hostname. cellucid.com’s Worker is served at https://api.cellucid.com (its
GitHub App callback is https://api.cellucid.com/auth/callback); a
self-hosted deployment that wants a limit serves its Worker on its own
Cloudflare domain the same way and adds a WAF rate-limiting rule for /cap/*
there
(Workers limits, rate limiting rules).
Configure the GitHub App with Expire user authorization tokens kept on (GitHub’s default for a new app; an existing app opts in under Optional Features → User-to-server token expiration). The Worker accepts only an expiring user token and refuses a token that never expires, so sign-in fails with token expiration disabled. Also select Request user authorization (OAuth) during installation, disable webhooks, leave account permissions unset, and set these repository permissions:
Metadata: read-only
Contents: read and write
Pull requests: read and write
No Administration permission: Cellucid never creates a fork. Creating one through the API needs Administration write (GitHub: permissions required for GitHub Apps), so a contributor without write access forks the repository on GitHub themselves, and Cellucid finds that fork when they publish.
Two values are compiled into the Cellucid client, so a self-hosted deployment builds its own copy of the site:
the Worker origin,
DEFAULT_WORKER_ORIGINinassets/js/app/community-annotations/github-auth.js. A different runtime override is accepted only from a recognized local-development host. The exact Cellucid page origin must also be present inALLOWED_ORIGINS.the GitHub App’s slug in the Add repo link of the connection wizard,
https://github.com/apps/cellucid-community-annotations/installations/new(GITHUB_APP_INSTALL_URLinassets/js/app/ui/modules/community-annotation/github-connection-flow.js; Publish’s Install the app link uses it too). Set it to your own app’s slug; otherwise Add repo opens the installation page of cellucid.com’s app, not yours.
For commands, precise origin rules, the complete health JSON, and live
verification, follow cellucid/docs/github-oauth-cloudflare-setup.md in the
Cellucid web repository. A source edit does not update a running Worker; deploy
the accepted Worker source and then verify the live root and browser lifecycle.
9) Tune Consensus Rules Per Column#
Inside Annotated columns, after you select a column that is already annotatable, you will see Consensus settings:
Threshold slider (maps to
thresholdinannotations/config.json)Min annotators input (maps to
minAnnotators)Apply (apply locally)
Reset (discard local edits)
Consensus settings appear only once you select a column that is already annotatable — the block belongs to the selected column, not to the repository. The Threshold slider is in whole percent; the stored value is a signed fraction spanning −1 to +1, because confidence itself is signed.#
After applying locally, Publish to write the settings to annotations/config.json so all annotators receive them on Pull.
Settings created when a field is enabled#
The keys in annotatableSettings must be exactly the same keys as
fieldsToAnnotate. When an author enables a field in the UI, Cellucid creates
its required entry with:
minAnnotators = 1threshold = 0.5
You may then edit these values before Publish. A config file may not omit the settings entry for an enabled field.
Worked examples (to build intuition)#
Per category bucket, Cellucid computes:
voters= number of unique users who voted on any suggestion in this categorynetVotes= (upvotes - downvotes) for the current leading suggestionconfidence = netVotes / voters
Examples:
Votes in the bucket |
voters |
Best netVotes |
confidence |
Notes |
|---|---|---|---|---|
1 user upvotes “B cell” |
1 |
1 |
1.0 |
Still Pending if |
3 users upvote “B cell”, nobody downvotes |
3 |
3 |
1.0 |
Strong consensus |
3 users: 2 upvote “B cell”, 1 downvotes “B cell” |
3 |
1 |
0.33 |
Often disputed unless threshold is low |
4 users: 3 up, 1 down |
4 |
2 |
0.5 |
Exactly meets |
4 users split: 2 upvote A, 2 upvote B |
4 |
2 |
0.5 |
Disputed because net votes and upvotes both tie |
Important
threshold is not “percent upvotes”.
It is a net-support share: (upvotes - downvotes) / voters.
Downvotes matter and reduce confidence.
Practical parameter choices#
Starting points:
Conservative rounds (high confidence):
minAnnotators=5,threshold=0.6–0.8Fast iteration rounds:
minAnnotators=2,threshold=0.5
If you expect disagreement (hard biology, rare types):
keep
minAnnotatorsrelatively high (so one loud person can’t dominate),use comments as evidence,
accept that some clusters will remain Disputed and require manual review.
10) Merge Duplicate Suggestions (Moderation)#
Even with careful contributors, duplicate labels happen (e.g. T cell vs T-cell, synonyms, or capitalization variants).
Suggestions with exactly the same label need no merge. Inside a category, Cellucid treats them as one suggestion: labels are compared after Unicode normalization (NFC), so the same text typed with composed or decomposed accents counts as equal, but nothing else is folded (case, spacing and punctuation still differ). The earliest proposal is shown as the suggestion, and each person’s votes across the equal labels count once (their majority; a tie counts as no vote). Cellucid also refuses to add, or rename a suggestion to, a label already shown in that category. Two people proposing “T cell” independently therefore support one label instead of splitting it into a tie.
For labels that differ but mean the same thing, an author merges them so votes are combined:
Open a category’s voting modal.
Drag one suggestion card onto another.
Confirm the merge and optionally write a merge note. Like comment edits, a merge-note edit in progress survives the modal refreshing around it.
Publish (authors publish to
annotations/moderation/merges.json).
Merge behavior (what annotators experience):
Votes are combined and de-duplicated (at most one vote per user in the merged bundle total).
The UI keeps originals accessible via View merged (so evidence and history aren’t lost).
When to merge vs when not to merge#
Merge when the suggestions are truly the same meaning:
formatting variants (
T cellvsT-cell)spelling variants
the same ontology id with different labels
one label is a strict synonym of the other in your project
Do not merge when meaning differs:
“CD4 T cell” vs “T cell” (one is more specific)
“Doublet” vs “Cycling T cell” (different biological claims)
unresolved debates (leave Disputed; use comments)
What is stored in merges.json (advanced)#
annotations/moderation/merges.json (version 3) is an author-only log of
merges: {"version": 3, "updatedAt": "<UTC timestamp>", "merges": [...]}.
Each merge entry includes:
datasetId: the dataset the merge belongs to. One repository can serve several datasets that share a field and a category label, so a merge applies only to the bucket of its own dataset.bucket: which category bucket the merge applies to. Its format is<fieldKey>:<categoryLabel>when the field key contains no colon. When it does, Cellucid usesfk~${encodeURIComponent(fieldKey)}:<categoryLabel>; thefk~prefix and percent-encoding make the embedded colon unambiguous.fromSuggestionId→intoSuggestionId: the mapping (merge “from” into “into”). Both are suggestion ids in the owner-prefixed formghid_<proposer id>_<rest>; they may name any contributor’s suggestions.by: author identity (stored asghid_<githubUserId>)at: timestampeditedAt(optional): if you later edit the merge notenote(optional): a short rationale shown in the UI
Example of one entry (illustrative):
{
"datasetId": "my_atlas_v1",
"bucket": "leiden:7",
"fromSuggestionId": "ghid_7_6f1c2c0e-3d1b-4e37-9a83-2f0b54c7d1a9",
"intoSuggestionId": "ghid_8_0b9e4a51-7c2d-4f8e-b3a6-5d1e9c2f7a40",
"by": "ghid_12345",
"at": "2025-01-01T12:34:56Z",
"note": "Merged formatting variants: 'T cell' and 'T-cell'."
}
Undoing a merge (if you merged the wrong thing)#
Open View merged on the target suggestion. Authors can detach a merged
member there. Only the GitHub identity that created a merge record can edit or
delete its note. Publish afterward to write the changed merge log. For recovery
outside the UI, revert the commit that changed
annotations/moderation/merges.json; direct JSON editing is advanced and must
be validated before Pull.
11) Build a Consensus Column (Optional, Local)#
Cellucid can build a local derived categorical obs column for visualization:
Open Consensus column.
Choose an annotatable source column (e.g.
leiden; a column you renamed here is listed by its name here, and its votes are read under its dataset name).Choose a New column key (it starts as
community_<column>).Threshold / min annotators start at the column’s rule in the repository (
annotations/config.json); a change applies to this derived column only and is remembered for the column.Build the derived column.
Result:
Each category becomes a label (if consensus), or
Disputed, orPending.This does not change voting rules and does not publish anything to GitHub.
A built column is a step of the history (field.consensus), like any other
change: Undo takes it back, and a session or a shared history carries it. The
step records, for each category of the source column, its outcome (the agreed
label, disputed or pending, with its voter count), the rule it was built with
(threshold and minimum annotators) and where the votes came from (repository,
branch and when it was last pulled); the cells’ labels are derived from the
source column when the step is applied. So it replays by name: on another
dataset with a column and categories of the same names, each category gets
its recorded outcome. A step recorded before this version kept one code per
cell, so it is put back only on the dataset it was built on.
A column built is a snapshot of the consensus at that moment. When a Pull (or
a vote) changes the consensus of its source column, the panel says
<column>: consensus changed since built (N categories) with Rebuild,
which builds it again under its name with the rule it was built with.
A category agreed as literally Pending or Disputed stays a label of its
own; the status category is then named Pending (status) (or Disputed (status)).
Categories that agree on the same label are one category of the derived column, compared as suggestions are (after Unicode NFC normalization; case, spacing and punctuation are not folded), and it is spelled as the first of them spells it.
This is useful to:
color the atlas by the current community consensus
quickly spot which clusters still need attention
Note
The derived column uses the threshold/minAnnotators set in the derived-column UI (the repository’s rule unless you changed it); the history step records them, and the methods paragraph of the history states them.
12) Export a Consensus Snapshot (For Downstream Tools)#
Inside Snapshot and cache:
Snapshot and cache → Download (
cellucid-consensus.json)
This produces a JSON snapshot built in your browser from the locally cached raw GitHub files (it is not written back to GitHub).
Recommended author workflow:
Pull latest (ensure you have everyone’s newest files)
Download
cellucid-consensus.jsonUse it downstream (e.g., build an official cell-type column)
What is inside cellucid-consensus.json? (Structure)#
The snapshot contains:
suggestions: merged suggestion cards per bucket, includingupvotes/downvotesarraysconsensus: per-bucket summary objects with:status:pending|disputed|consensuslabel: best label (or comma-joined labels in ties)confidence: number in -1..1voters: unique voter countnetVotes: best net vote countsuggestionId: winning suggestion id (null in ties)
Buckets use <fieldKey>:<categoryLabel> when the field key contains no colon.
For a field key containing :, Cellucid uses
fk~${encodeURIComponent(fieldKey)}:<categoryLabel>. The category label
follows the first delimiter verbatim. Treat fk~ as the escape prefix only
with %3A in the prefixed component; a colon-free field key may itself
begin with fk~.
Example downstream usage (computational)#
This is one simple pattern: map consensus labels back onto an AnnData cluster column.
import json
from urllib.parse import unquote
import pandas as pd
doc = json.load(open("cellucid-consensus.json"))
target_field = "leiden"
mapping = {}
for bucket, summary in doc["consensus"].items():
field_key_part, category_label = bucket.split(":", 1)
encoded_field_key = field_key_part[3:]
field_key = (
unquote(encoded_field_key)
if field_key_part.startswith("fk~") and "%3a" in encoded_field_key.lower()
else field_key_part
)
if field_key != target_field:
continue
if summary.get("status") == "consensus":
mapping[str(category_label)] = summary.get("label")
# Example: apply to a Series (or adata.obs[target_field])
cluster = pd.Series(["0", "1", "7", "7"], name=target_field)
cluster_consensus = cluster.astype(str).map(mapping)
Notes:
For disputed or pending buckets, choose an explicit display label such as
"Disputed", or leave the value missing.If your cluster labels are integers, cast consistently to strings.
Limits (What Cellucid Cannot Do or Check)#
These come from GitHub, Cloudflare or the browser, not from Cellucid. They are listed so a large round can plan around them.
One file per contributor, at most 1,000,000 UTF-8 bytes. GitHub’s Contents API returns a file’s content as JSON only up to 1 MB (GitHub: get repository content). A contributor’s one file holds every dataset they annotate in the repository.
Listing of
annotations/users/. GitHub documents a limit of 100,000 entries or 7 MB for a tree listing (GitHub: get a tree). When GitHub reports the listing as truncated, Pull stops and says so rather than compiling the part it saw.GitHub’s hourly budgets. Every request counts against the signed-in user’s REST and GraphQL budgets (REST, GraphQL). Cellucid paces by what GitHub reports and waits when GitHub says so; frequent auto-pulls by many people spend each person’s own budget.
Direct-mode trust. Anyone with push access can write any file with Git or GitHub’s web editor, including another contributor’s
annotations/users/ghid_<id>.json. The Worker refuses such a write made through Cellucid, but Cellucid reads files by content and cannot verify who committed them. Review the commit history when in doubt.The Worker cannot bind a write to the connected branch. Which branch a page has connected lives in the browser; the Worker binds every write to the signed-in user (their own file, their own
cellucid-annotations/ghid_<id>/branches), not to that branch.Cloudflare Workers Free plan (cellucid.com’s Worker and the checked-in configuration): about 10 ms of CPU and 50 subrequests per request (Cloudflare: Workers limits). The checked-in Worker stays well inside both (it relays reads without parsing them, and a 1,000,000-byte publish costs a few milliseconds). A request over the CPU budget is answered by Cloudflare with an HTML
503page without CORS headers, which the browser can only report as an unreachable Worker; Cloudflare’s logs show it asexceededCpu(the Worker setup guide’s failure diagnosis covers it).