Open R objects and MatrixMarket folders#
Audience: anyone with a Seurat or SingleCellExperiment object saved from
R, or a count matrix in MatrixMarket form
Time: 5 minutes
What you’ll learn: how to open an .rds, .rda or .RData file and a
MatrixMarket folder directly, what Cellucid reads from each and what it
leaves out, how long a large object takes, and what each refusal asks you to
do
Prerequisites: none
Cellucid opens these files as they are: there is no “convert to AnnData
first” step. Unlike an .h5ad, a compressed R file or a gzip text file has
no random access, so it cannot be read in parts: it is read whole, once,
from the start into this browser’s storage (the origin-private file system),
the expression matrix laid out by gene and the cells in priority order as a
Cellucid-ready AnnData’s are (so a subsample opened later reads the first rows
of every array), and
every view afterwards reads what was kept there, as it reads any AnnData
store (File format expectations). The opening says so before it
reads, with how much it reads. What is kept stays for your next visit when
the file has a version (a file on this computer always has one; a linked
file when its host reports one): the same file opened again, in this tab or
another, today or next week, opens from there in seconds and reads nothing of
the file (Which formats are read in parts). Of a file known only by
its size nothing is used by a later opening: it is read again.
Seurat and SingleCellExperiment objects (.rds, .rda, .RData)#
Open… › File…, drop the file on the viewer, or paste a link to it
(Open from a link; a linked R file is read as it downloads, and
nothing is saved to your Downloads folder). A gzipped copy (.rds.gz,
.rda.gz, .RData.gz, as GEO stores R objects) opens the same way: the
file’s content, not its name, says how many times it is compressed, and it
is decompressed as it streams. A file dialog that matches only a file’s last
extension may grey out an .rds.gz; drop it on the viewer instead.
Try it with SeuratObject’s own sample#
In R:
library(SeuratObject)
saveRDS(pbmc_small, "pbmc_small.rds")
Open pbmc_small.rds. Expected: 80 cells, 230 genes, the embeddings
X_pca and X_tsne, the obs fields of pbmc_small[[]] (orig.ident,
nCount_RNA, RNA_snn_res.0.8, letter.idents, groups, …) plus
active_ident, and the RNA_snn neighbour graph. The Dataset box says
Read whole once (it cannot be read in parts), kept in this browser’s storage
for next time · 5 parts not read:
pbmc_small carries a scale.data matrix, PCA loadings and JackStraw
results, and its commands, which Cellucid leaves out
(below).
A SingleCellExperiment saved the same way (saveRDS(sce, "sce.rds")) opens
the same way.
What is read#
Seurat (SeuratObject 3.0 and later) |
SingleCellExperiment (1.8 and later), SummarizedExperiment |
In Cellucid |
|---|---|---|
|
|
the cells and their obs fields |
|
the obs field |
|
|
|
the embedding |
the default assay’s |
|
the expression matrix |
|
|
the gene table; |
other assays ( |
|
modalities (Multimodal data: MuData and the Modality select), the default assay first |
|
|
neighbour graphs; the default assay’s |
Visium and FOV images |
the cells’ positions as the embedding |
|
|
|
what the dataset says about itself ( |
|
the dataset’s title (unless |
Seurat v5 objects with split layers (counts.1, data.ctrl, made by
split() or merge()) are joined as JoinLayers() would. Factors keep
their level order and NA stays missing; logical and integer columns with
NA stay nullable.
Left out, and never downloaded twice: scale.data (dense, the variable
genes only), commands, tools, neighbors, feature loadings, JackStraw
results and fitted models. The file still has to be read past them once, but
nothing of them is kept, not even a Seurat v5 scale.data layer, whose name
R writes only after its values.
Each one that holds something is listed in the Dataset box’s Details under
Not read, with its size in the file; the empty scale.data, commands
and neighbors every Seurat object carries are not.
Cellucid reads R’s own serialization (versions 2 and 3, gzip, bzip2, xz or
zstd (saveRDS(compress = "zstd"), R 4.5 and later; decoded by zstd itself,
in WebAssembly), or uncompressed) with its own parser in the browser: R is not needed. An
uncompressed file (saveRDS(obj, file, compress = FALSE), or
readr::write_rds(), which does not compress) is not copied at all: its
structure is read and every matrix is then read in place from the file, as
from an .h5ad. Its cells therefore keep the object’s order (a converted
file’s are put in priority order), and a subsample reads it in blocks, as any
file read in place.
A file with several objects#
A .rda or .RData file that holds more than one Seurat object or
SingleCellExperiment (save(seu_all, seu_tcells, file = "objects.rda")) is
read once, keeping every one of them, and the opening asks which to open:
objects.rda holds 2 single-cell objects. Open which one?
seu_all (Seurat, 52,113 cells) · seu_tcells (Seurat, 8,201 cells)
An .rda holding two Seurat objects, opened from this computer: the file is
read once, then the opening asks which object to open, with each one’s class
and cell count. Cancel keeps the dataset that was open.#
Opening the same file again and choosing the other one reads nothing
again. Other objects in the file (data frames, vectors) are passed
over as the file streams, nothing of them kept: a Seurat object, a
SingleCellExperiment and a SummarizedExperiment are all S4 objects, and the
first word of each object in the file says whether it is one. An .rds
whose object is not (a data frame of counts, a matrix, a list) is refused
from its first bytes, without downloading the rest (GEO’s 604 MB
GSE131907_Lung_Cancer_raw_UMI_matrix.rds.gz was refused only after its
whole download, 194 s).
How long a large object takes#
The first opening runs at the speed of the slower of the link and the
decompression; every later one opens what was kept. Measured on an Apple M1
Pro (2026-10-27) with Seurat v5 objects (counts and data, PCA, UMAP, an SNN
graph, meta.data; scripts/rds-corpus/make_bench.R):
Object |
First opening |
Later visits |
A Cellucid-ready AnnData of it |
|---|---|---|---|
50,000 cells, 1,500 genes per cell, with |
linked over 25 MB/s and 40 ms: 56 s to the first view, 1.2 GB downloaded, 1.6 GB peak browser memory; from disk (Node): 11 s |
1.2 s to the first view, nothing downloaded; a gene in 0.3 s |
244 MB |
300,000 cells, 500 genes per cell; 1.05 GB gzip |
linked over 25 MB/s: 55 s, 1.05 GB, 2.1 GB peak; from disk (Node): 19 s |
1.2 s, nothing downloaded |
|
the 50,000-cell object as xz / bzip2 (0.98 GB / 0.99 GB) |
from disk (Node): about 200 s / 115 s (the decompression itself, about 13 / 23 MB/s) |
1.2 s |
A Seurat object stores its assays first and its reductions near the end
(the 50,000-cell object’s embedding starts after 98 % of the file), so no
view can be drawn before the whole file is read. The pass holds a bounded
window in memory, whatever the object’s size (under 700 MB in Node for
either object). While it reads, the opening shows how much is read, how
fast and the time left, with Cancel; a cancelled pass leaves nothing
behind. To share a large object with others, write it once as a
Cellucid-ready AnnData with cellucid::cellucid_prepare(path, data = obj)
(R) or cellucid.prepare() (Python): that is read in place, so a viewer
reads only what a view needs.
A linked Seurat .rds opening while no other dataset is on screen: the panel
over the empty viewer names the file, the step under way (here a host that
sends whole files only) with its size, how much has arrived, the speed and the
time left, then, before anything else, why the file is read whole, how much is
read, that it is kept for next time and what is read in parts instead, and
Cancel. With a dataset on screen the same lines are a
notification, and that dataset stays until the new one is ready.#
MatrixMarket folders#
Open… › Folder… (or drop the folder) on a folder holding a MatrixMarket count matrix and the two lists that name its rows and columns:
Written by |
Matrix |
Features |
Cells |
|---|---|---|---|
Cell Ranger 3 and later |
|
|
|
Cell Ranger 2, STARsolo |
|
|
|
kb-python |
|
|
|
alevin-fry |
|
|
|
a GEO upload |
|
|
|
sci-RNA-seq3 (GEO GSE119945, GSE186069, GSE228590) |
|
|
|
A folder is a MatrixMarket folder when it holds a .mtx file (.mtx.gz,
.bz2, .xz; any case), whatever its producer named it. A sample prefix
(GSM5008737_RNA_3P-, ending in _, - or ., before the matrix’s last
word) pairs each matrix with its own lists, so a folder of a GEO series’ extracted
_RAW.tar, every sample side by side, opens too: when it holds several
samples Cellucid asks which one to open. Lists under a shorter shared prefix
serve every matrix below it (one GSE1_barcodes.tsv.gz for
GSE1_RNA_matrix.mtx.gz and GSE1_ADT_matrix.mtx.gz), but never a matrix
that has lists of its own. A sample whose barcode or feature list is missing,
or that has both a features and a genes list, is refused naming the
files. A matrix has no embedding: one of the folder’s cell tables must name
one (UMAP_1/UMAP_2 columns, Cell Ranger’s projection.csv). Whether one
can is told from each table’s first lines, so a folder none of whose tables
names an embedding is refused before its lists, tables or matrix are read
(GEO’s 1.46M-cell GSE158055: refused in about 2 s after under 1 MB, instead of
reading 15 MB of lists and tables and part of an 8.2 GB matrix). A GEO sample can also be opened from its link without downloading it
(Open from a link).
A list’s first line is its header when the list has one line more than the
matrix’s size line says it names, whatever the line says; when its length
tells nothing, only when it is a column name and nothing else (barcode,
cell_id, gene, gene_id, feature, id, Parse’s bc_wells): a first
cell named cell0 or a first gene named IDH1 is a cell or a gene. A
feature list with a header names its columns: the gene names are read from
gene_short_name, gene_name, symbol or name, the ids from gene_id
or id, and the modalities only from feature_type (a biotype such as
gene_type is a column of the genes, not a modality). A cell list that is a
table of the cells (a sci-RNA-seq3 cell_annotate.csv, Parse’s
cell_metadata.csv) is read once, as the list and as a table: its columns
are obs fields and its embedding columns (tsne_1/tsne_2) embeddings.
GEO GSE119945’s matrix is named …_gene_count.txt.gz, MatrixMarket under a
.txt name. A link and a folder picked from this computer tell it by the same
names: a …_gene_count.txt.gz (or .run_13.txt.gz) is a matrix, and its two
tables go with it. Without them it is refused, naming the missing table
(…_gene_annotate.csv.gz, …_cell_annotate.csv.gz). Any other .txt file
is not a matrix. cellucid serve tells such a folder by the same rule.
The matrix may be genes × cells or cells × genes; which it is is read from
its size against the two lists. A filtered matrix is preferred over a raw
one in the same folder. The gene table has gene_ids and, for Cell Ranger 3
and later, feature_types. Features of several types (features.tsv
column 3: Gene Expression with Peaks in a Multiome sample, or with Antibody
Capture) open as one modality per type, Gene Expression first, as a 10x
.h5 does (Multimodal data: MuData and the Modality select). A matrix written by cell (Cell
Ranger, STARsolo) or by feature (kb-python, alevin-fry) is read as it was
written and one in any other order is sorted in blocks, so the browser’s
memory never has to hold the whole matrix.
The cells first, the matrix after them#
The lists and the cell tables are small; the matrix is nearly all of the bytes, and one gzip stream that can only be read from its start (GEO GSE119945: a 106 MB cell table, a 3.95 GB matrix). So a folder whose matrix is larger than its lists and tables opens in two parts:
The cells are drawn first, from the lists and tables alone: every cell, its embeddings and its obs fields. A gene has no value yet: the legend says genes are read for none of the cells, and a notification says the matrix being read (its size, how far it is).
The matrix is read after it, in the reader worker, as it arrives, keeping only the rows of the cells genes are read for: the first cells in priority order, as many as this device’s memory holds a gene index of (the same rule as a gene sample of a matrix stored by cell, Subsampled loading: open a share of the cells), every cell when they all fit. What is kept follows that share, never the file: on a 4 GB device GSE119945’s 2.06 million cells keep 165,515 (8 %), about 0.8 GB, where every cell’s matrix would be 10 GB. Once it is read, genes colour those cells, analyses use them, and the other cells show no value until Genes read for (Session › Subsampling) reads the matrix again for more (its cost is said first).
Reading the share reads the matrix to the share’s last cell: a matrix written by cell is left there, one written by gene (kb-python, alevin-fry) or in no order is read to its end. The share’s cells are spread over the whole file (a uniform sample: a share of the first cells in the file would be the first samples or donors only), so a share reads nearly all of a matrix written by cell too: its time is the matrix’s download, whatever the share. So the matrix is read once an opening: a folder whose cells open progressively (a first view, then more, Subsampled loading: open a share of the cells) reads it once, for the cells it grows to, never again a step; when more is asked for, the larger share replaces the one kept, and a folder keeps one. A matrix no larger than its lists and tables, or one of several modalities (whose genes are read for every open cell), is read with the cells, and the folder opens whole.
A gzip stream has no index, and one cannot be made without reading the stream: starting in the middle of one (finding a block start by trial and decoding without the 32 KB that came before it) leaves bytes unknown until everything copied from before the start has been written over, and on these files that is never, or only after megabytes (GSE158055: a quarter of the bytes still unknown after 20 MB of the stream, on every line; GSE119945: 1 to 6.5 MB). So the matrix is read from its start.
Both parts are kept in this browser for later visits, under the files’ versions: a later visit draws the cells and reads genes from what was kept, without reading the folder again. A visit left while the matrix is read keeps the cells, not the part of the matrix read so far: the next visit reads the matrix again.
A count matrix has no cell embedding, and Cellucid draws cells at embedding coordinates. The folder’s own cell tables give it one:
Cell Ranger’s
analysis/folder:analysis/umap/*/projection.csv,tsne,pca(asX_umap,X_tsne,X_pca) andanalysis/clustering/*/clusters.csv(as obs fields named after the clustering);any
.csvor.tsvtable of the cells in the folder (cell_metadata.csv, a GEOmetadata.tsv): its cell column is matched to the barcodes, columns named likeUMAP_1/UMAP_2ortSNE-1/tSNE-2become embeddings and the others obs fields. A column named for a clustering (Cluster,seurat_clusters,leiden,louvain,kmeans_10,ClusterID) is categorical even when its labels are numbers, as Cell Ranger’sclusters.csvis, and as a Loom file’s integer clustering is.
Without either the folder is refused with the lines that compute an
embedding in Python and in R. A Cell Ranger outs/ folder that also holds
filtered_feature_bc_matrix.h5 opens through that file instead, read in
place (Open Loom and 10x Cell Ranger files).
A Cell Ranger 3 MatrixMarket folder with a cell_metadata.csv beside it,
opened with Open… › Folder…: the table’s UMAP_1/UMAP_2 columns are the
embedding and its other columns obs fields. Like an R object, the folder is
read whole once and kept in this browser’s storage for next time.#
What the Dataset box says#
Under the line that names the format and where it is read from (R object (.rds) · this computer, MatrixMarket folder · this computer), one more line says where the converted data is kept and how many parts of the file were not read:
The line |
When |
|---|---|
Read whole once (it cannot be read in parts), kept in this browser’s storage for next time |
the first opening of the file |
Opened from this browser’s storage, read whole on an earlier open |
the same file opened again (a file with a version: in any tab, on any later visit; one known only by its size: in its own tab): nothing of the file is read |
Read whole once (it cannot be read in parts), into this browser’s storage for this tab |
a file known only by its size (a link whose host reports no version): used by this opening only, and read again by the next. A viewer host’s file ( |
Structure read once (arrays read in place), kept in this browser’s storage for next time |
an uncompressed R file ( |
Read whole once (it cannot be read in parts), into memory for this tab |
the browser gives the page no storage (a Firefox or Safari private window, a page not served over HTTPS): the converted data is held in memory instead, and is read again on each opening |
followed by · 5 parts not read when something was left out. The Details disclosure says the rest:
Read: Whole, once, from start to end (it cannot be read in parts), from this computer; nothing is uploaded, or … from the host of a link;
Kept: where the converted data stays (this browser’s storage, the origin-private file system, until it is cleared or the browser needs the room; or memory, until the tab closes);
Read in parts: what is read in parts instead, a Cellucid-ready AnnData written with
cellucid_prepare()(R) orcellucid.prepare()(Python);Not read: one line per part left out and why, named by its place in the R object, for example assays$RNA@scale.data (15 KB): Cellucid does not use it., assays$ADT@counts is kept outside the file (MatrixDir at …) and was not read., or a cell table of a MatrixMarket folder that could not be read.
An .h5ad, an AnnData Zarr store and the other formats read in place have
none of these: nothing of them is copied.
Session › Stored sums up what is kept in this browser and how much it takes; Manage… lists it by dataset (its file’s name and host), each kind kept with its size and when it was last used, and removes the ones you choose, a dataset’s whole, or all of them, that no tab has open (Stored in this browser). When a new file does not fit in the storage the browser grants the site, kept gene indexes and downloaded parts are removed first, then the files opened least recently; a gene index being kept never removes a converted file.
Everything kept is written with a checksum of every 16 KB of it (CRC-32), and every later read of it is checked against them: converted data, and a file downloaded whole from a host that serves no parts of files. Damaged bytes are never shown: the read fails saying the kept copy is damaged, the copy is removed, and opening the file again reads it again.
A Seurat object saved with saveRDS() and opened from this computer, with
Details open: Read says it was read whole, once, from start to end and
nothing was uploaded, Kept that the converted data stays in this browser’s
storage until it is cleared, and Read in parts what is read in parts
instead.
This object carries nothing Cellucid leaves out, so no parts not read follow.#
When a file is refused#
Each refusal says what the file is and what to do; the sections of Troubleshooting (data loading) say more.
Message says |
Why |
What to do |
|---|---|---|
… keeps the RNA assay’s counts outside the file (MatrixDir at …) |
a BPCells or HDF5Array matrix: the values are not in the |
in R, |
… has no layers in the file (SaveSeuratRds() moved them …) |
|
load it with |
This is a Seurat v2 object |
Seurat 2 (2017–2018) classes |
|
This is an h5Seurat file |
SeuratDisk’s own format, not read |
|
… has no cell embedding |
no reduction, no |
run PCA and UMAP ( |
… saved as text / in R’s native binary form |
|
save with the defaults |
… holds an R data frame (or an R list with a class (a data frame, for one), an R numeric vector or matrix), not a Seurat object or a SingleCellExperiment |
the file holds some other R value; an |
save the single-cell object itself; a count matrix opens once it is part of one with an embedding ( |
This MatrixMarket matrix is symmetric / complex / a dense array |
not a count matrix |
write the counts as a coordinate matrix |
.qs and .qs2 files (the qs packages) are not R serialization: save the
object with saveRDS(obj, file).