JSONL, shards, and the path to the GPU: from one line to training
How large training sets are laid out: JSON vs JSONL, shards, compression, tokens, batches, and epochs — and why a terabyte of data does not need a terabyte of RAM.
Stuzhuk Lab — Chemistry of Code
Chemistry of Code
← All posts
Tag
ml dataset engineering
All blog posts with this tag.
SFT datasets: how to design a supervised fine-tuning corpus
A pillar guide to supervised fine-tuning corpora: instruction→answer contracts, coverage, refusals, synthetic vs expert sources, pre-train gates, and eval leakage.
Dataset engineering for model training: where to start
A pillar guide to data for training and evaluating AI: golden records, golden editors, train/eval splits, leakage, manifests, and parameters by task and domain.