Data Formats Reference
FASTQ, SAM/BAM, GTF, and VCF are file formats for storing and exchanging biological information. This reference groups the formats commonly encountered in personal genomic and transcriptomic analysis into two tracks: RNA expression and DNA variants.
Shared: raw output from the sequencer
Section titled “Shared: raw output from the sequencer”The sequencer’s raw reads and their quality scores. Both RNA-seq and DNA sequencing start here. One read occupies four lines.
@SEQ_IDGATTTGGGGTTCAAAGCAG+!''*((((***+))%%%++BAM / SAM
Section titled “BAM / SAM”Reads aligned to a reference genome or transcriptome. SAM is a text format and BAM is its compressed binary representation. Reading One SAM/BAM Line explains the structure of each record.
An annotation file that records gene, transcript, and exon coordinates on a genome. In RNA-seq, it helps determine which gene receives an aligned read. See the GTF reference for details.
RNA track: expression
Section titled “RNA track: expression”Gene expression matrix
Section titled “Gene expression matrix”A main output of bulk RNA-seq. Rows are genes, columns are samples, and each value represents expression.
A bulk RNA-seq matrix made from mixed tissue reports RNA pooled across the entire tissue piece, not a separate value for each cell.
gene control_1 control_2 treated_1gene_A 842.1 93.4 88.0gene_B 210.5 198.2 205.7| Unit | Meaning |
|---|---|
| counts | An unnormalized value obtained by assigning aligned reads to a gene and counting them |
| CPM / TPM / FPKM | A value normalized by library size and gene length, often used for comparison across samples |
Quantification tools include
featureCounts,Salmon, andkallisto. Aligners explains the differences among alignment tools such asSTARandHISAT2.
Single-cell formats
Section titled “Single-cell formats”Because individual cells are measured separately, the matrix is large and sparse. A 10x MEX file stores features × barcodes, while AnnData usually exposes cells × genes, so always inspect the axes.
| Format | Contents |
|---|---|
.h5ad (AnnData) | A common Python (scanpy) format containing the matrix plus cell and gene metadata |
.rds (Seurat) | A standard object in the R ecosystem |
| 10x MatrixMarket | Three files: matrix.mtx, barcodes.tsv, and features.tsv |
Common tools include
scanpyfor Python andSeuratfor R.
Single-Cell Data Structures and File Formats explains raw and filtered matrices, MEX, H5, h5ad, Seurat objects, and their axes and slots.
DNA track: variants
Section titled “DNA track: variants”VCF (Variant Call Format)
Section titled “VCF (Variant Call Format)”A list of variants relative to a reference. VCF is a central format in whole-exome sequencing (WES) and whole-genome sequencing (WGS) analysis.
#CHROM POS ID REF ALT QUAL FILTER INFO1 11856378 rs1801133 G A 99 PASS ...Use bcftools to filter, query, and merge VCF files.
SNP-chip raw data (23andMe or laboratory text files)
Section titled “SNP-chip raw data (23andMe or laboratory text files)”Tab-separated text containing the genotypes of hundreds of thousands of variants measured by a SNP chip. This is one of the most common forms of raw data returned by a testing provider.
| Column | Meaning |
|---|---|
| rsid | A dbSNP variant identifier such as rs... |
| chromosome | Chromosome |
| position | Coordinate relative to a specific genome build, often GRCh37 |
| genotype | The two alleles, for example AA or AG |
Key public databases
Section titled “Key public databases”| Database | Use |
|---|---|
| GEO | Gene-expression data and experimental metadata organized by study |
| ENA | Public raw sequencing data such as FASTQ files |
| GTEx | Normal expression by tissue, providing a reference for whether expression is high or low |
| dbSNP | Variant identifiers and basic information |
| ClinVar | Clinical interpretation, including pathogenicity |
| gnomAD | Allele frequencies in populations |
| SNPedia | Community-contributed phenotype interpretations |