Data Formats Reference
A summary of the file formats you’ll meet most often in personal genomics and transcriptomics. Following Sid’s case, we start from expression (RNA) and move toward variants (DNA). The formats are grouped into those two tracks.
Shared: the sequencer’s raw output
Section titled “Shared: the sequencer’s raw output”The raw reads a sequencer produces, with quality scores. Both RNA-seq and DNA sequencing start here. Four lines per read.
@SEQ_IDGATTTGGGGTTCAAAGCAG+!''*((((***+))%%%++BAM / SAM
Section titled “BAM / SAM”Reads aligned to a reference (genome or transcriptome). SAM is text, BAM is its compressed binary form. Handled with samtools.
RNA track: expression
Section titled “RNA track: expression”Gene expression matrix
Section titled “Gene expression matrix”The final output of bulk RNA-seq. Rows = genes, columns = samples, values are expression.
gene tumor_01 tumor_02 normal_01MDM2 842.1 93.4 88.0TP53 210.5 198.2 205.7| Unit | Meaning |
|---|---|
| counts | Number of reads aligned to a gene (raw) |
| CPM / TPM / FPKM | Normalized by library size and gene length (for comparing across samples) |
Quantification tools:
featureCounts,Salmon,kallisto. Alignment:STAR,HISAT2.
Single-cell formats
Section titled “Single-cell formats”Because each cell is read separately, the matrix is huge and sparse: usually cells × genes.
| Format | Contents |
|---|---|
.h5ad (AnnData) | The Python (scanpy) standard. Matrix plus cell/gene metadata in one file |
.rds (Seurat) | The standard object in the R ecosystem |
| 10x MatrixMarket | A set of three: matrix.mtx + barcodes.tsv + features.tsv |
Tools:
scanpy(Python),Seurat(R). Sid’s ①③ treatments and the 19%→89% T-cell analysis came from this data.
DNA track: variants
Section titled “DNA track: variants”VCF (Variant Call Format)
Section titled “VCF (Variant Call Format)”A list of variants relative to a reference: the central format for whole-exome (WES) and whole-genome (WGS) analysis.
#CHROM POS ID REF ALT QUAL FILTER INFO1 11856378 rs1801133 G A 99 PASS ...Filter, query, and merge with bcftools.
SNP-chip raw (23andMe / lab txt)
Section titled “SNP-chip raw (23andMe / lab txt)”Tab-separated text. Genotypes for the hundreds of thousands of variants a SNP chip measures: the most common raw data a testing lab returns.
| Column | Meaning |
|---|---|
| rsid | dbSNP variant identifier (rs...) |
| chromosome | Chromosome |
| position | Coordinate (relative to a genome build, usually GRCh37) |
| genotype | The two alleles (e.g. AA, AG) |
Key public databases
Section titled “Key public databases”| DB | Use |
|---|---|
| GTEx | Per-tissue normal expression reference (the baseline for “high/low” expression) |
| dbSNP | Variant identifiers and basic info |
| ClinVar | Clinical significance (pathogenicity) |
| gnomAD | Allele frequency in populations |
| SNPedia | Community phenotype interpretation |