What these tools are

PyTorch: tensors, autograd, torch.nn, optimizers, Dataset/DataLoader, CUDA/devices, distributed training, a mature research ecosystem, and workable production paths via export.

TensorFlow: tensor computation, differentiation (GradientTape), Keras, tf.data, GPU/TPU, distributed training, and a strong deployment branch (Serving, Lite, and related tools).

Separate the layers upfront:

PyTorch / TensorFlow  →  ML frameworks (training)
YOLO / Transformers / Diffusion  →  model ecosystems on top of frameworks

A short timeline (no museum tour):

2015  TensorFlow
2016  PyTorch
2017+ both ecosystems grow
2020+ PyTorch especially visible in modern DL research
2020+ TensorFlow strengthens production / Keras / TPU

Different history means different habitual API emphases, not “one is obsolete.”

Architecture: two views of one stack

PyTorch (simplified):

torch → Tensor · autograd · nn · optim · Dataset/DataLoader · device

Minimal model:

model = nn.Sequential(
    nn.Linear(10, 32),
    nn.ReLU(),
    nn.Linear(32, 10),
)

Layers are ordinary Python objects; forward runs when you call model(x).

TensorFlow (simplified):

TensorFlow → Tensor · GradientTape · Keras · tf.data · optimizers · GPU/TPU

The analogue via Keras is Sequential / Functional / subclassing keras.Model. High level: compile + fit. Low level: explicit GradientTape.

Tensors and automatic differentiation

A tensor is the central object: shape, dtype, device, operations.

# PyTorch
x = torch.tensor([1.0, 2.0, 3.0])

# TensorFlow
x = tf.constant([1.0, 2.0, 3.0])

A batch almost always adds a leading B axis. Without understanding shape ([B, C, H, W], etc.), comparing frameworks is pointless — the errors are the same.

Autograd / gradients:

PyTorch:     forward → loss → loss.backward() → .grad → optimizer.step()
TensorFlow:  forward → loss in GradientTape → tape.gradient(...) → apply_gradients

The math is one: backpropagation over a computation graph. What changes is syntax and when the graph is fixed for differentiation.

Historically TensorFlow was tied to a “static graph” and PyTorch to dynamic eager style. Today both ecosystems support eager and graph compile/acceleration; it is more useful to look at your code and API version than a 2017 meme.

Training loop and model definition

PyTorch — explicit loop:

for x, y in dataloader:
    optimizer.zero_grad()
    prediction = model(x)
    loss = criterion(prediction, y)
    loss.backward()
    optimizer.step()

Models are usually class MyModel(nn.Module).

TensorFlow — two floors:

# Keras: loop hidden
model.compile(optimizer=..., loss=..., metrics=...)
model.fit(train_ds, epochs=...)

# Custom loop: full control
with tf.GradientTape() as tape:
    prediction = model(x)
    loss = loss_fn(y, prediction)
gradients = tape.gradient(loss, model.trainable_variables)
optimizer.apply_gradients(zip(gradients, model.trainable_variables))

Models: keras.Model, Sequential, Functional API. Keras cuts boilerplate; a custom loop returns control close to PyTorch.

Practical tip: if you are learning the basics, write an explicit loop once on both stacks. When fit and DataLoader become transparent, the high-level API stops being a black box — and framework comparisons get fairer.

Data, accelerators, and performance

Data. PyTorch: Dataset + DataLoader (batch, shuffle, workers, transforms). TensorFlow: tf.data (map, batch, shuffle, prefetch) — a strong pipeline for GPU.

files → decode → transform → batch → prefetch → GPU

The bottleneck is often not “the other framework” but a GPU waiting for the next batch from disk.

Hardware. Framework ≠ hardware:

Python → Framework API → backend → CUDA / Metal / TPU runtime → hardware

NVIDIA CUDA is the main accelerator for both ecosystems. TPU is a notable plus for the TensorFlow/JAX branch. Apple Silicon — MPS in PyTorch and separate stacks like MLX. Device support depends on drivers and builds, not the logo.

Performance. You cannot honestly say “X is faster than Y” without a protocol. Speed depends on model, batch, input, GPU/memory, data pipeline, mixed precision, compilation, distributed setup, and implementation quality. A minimal reproducible benchmark: one model, one dataset, one batch, one machine, same epochs — measure samples/sec, epoch time, GPU memory.

Research, production, CV, and LLM

Research likes fast iteration and non-standard architectures — here the familiar eager style of PyTorch and the model zoo around it often win.

Production needs a stable path: training → export/optimization → inference runtime → monitoring. Both ecosystems can do it; the path may go past the training framework:

Training → Model → Export / Optimization → Inference Runtime

Computer vision. CNNs, detection, segmentation, OCR, CRNN, ViT live in both ecosystems. Many popular detection stacks (including the YOLO family) historically lean on PyTorch — an ecosystem fact, not “proof of superiority.”

LLM / Transformers. Hugging Face Transformers, fine-tuning, LoRA/PEFT, distributed training today show up more often in PyTorch-centric pipelines. TensorFlow backends and weights exist too; check the specific model card and guide, not a slogan.

Debugging. PyTorch is usually debugged like ordinary Python. Keras gives callbacks and a high level; with a custom loop, debugging becomes “explicit Python” again.

Deployment and ecosystems

TensorFlow inference branch: TensorFlow Serving, TensorFlow Lite, related formats for server/edge/mobile.

PyTorch branch: modern export mechanisms, often ONNX → ONNX Runtime, plus specialized runtimes. Browser path: ONNX → WebGPU / Transformers.js — see WebGPU + Transformers.js.

Do not mix up the training framework and the inference runtime: you can train in one and infer in another.

In a real project the framework is only part:

project/
├── data/ · models/ · training/ · evaluation/
├── inference/ · checkpoints/ · configs/

The framework gives tensors, layers, and a loop; data pipelines, metrics, configs, and serving are your engineering.

Another common surprise: a team “moves to PyTorch” while production still runs TF Serving or Lite — and lives in two worlds for months. That is fine if the bridge (export, I/O contracts, metric parity) is explicit. It is bad when the bridge is only implied.

One project two ways

Image classification (e.g. MNIST) on both frameworks:

PyTorch:     Dataset → DataLoader → Model → Loss → Optimizer → Training loop
TensorFlow:  tf.data → Model → Loss → Optimizer → fit / custom loop

Hands-on on this site: handwritten digits. After two implementations, compare not “elegance” but: lines to first training run, where the loop hides, how to save weights, how to run inference without training.

Summary table and decision map

Decision map (no ranking):

Understand backpropagation?     → micrograd
See how a framework is built? → tinygrad
CNN / CRNN / YOLO / OCR?      → often PyTorch ecosystem
Already on TF / need TPU?     → TensorFlow
Modern LLM pilots?            → often PyTorch (+ Transformers)
Inference in the browser?     → train anywhere → ONNX / Web

Link to learning frameworks:

micrograd → understand autograd
tinygrad  → see framework internals
PyTorch / TensorFlow → real development

More detail: ML frameworks. JAX sits nearby too (NumPy + grad/jit/vmap, strong on TPU) — the world does not end with two giants.

Myths and what sits underneath

• “PyTorch is research only” — oversimplified: applied CV/LLM and production via export are common.
• “TensorFlow is no longer needed” — also a myth if you have TF Serving, Lite, TPU, or a large Keras codebase.
• “They are the same thing” — no: different API style, different deployment history, different ecosystem center.
• “Choose for life” — engineers work with several stacks.
• “Framework = model quality” — no: data, architecture, training, and evaluation beat the logo.

Under one line loss.backward() (or tape.gradient) hides:

Python → Framework API → Autograd / graph → Tensor ops
       → Compiler / backend → CUDA/TPU runtime → hardware

FAQ

Further reading

• AI hardware
• ML frameworks for training neural nets
• ONNX: from PyTorch to Runtime
• WebGPU + Transformers.js
• YOLO engineering
• Compact CRNN
• Handwritten digits

Conclusion

PyTorch and TensorFlow are full frameworks for tensors, gradients, and training. They are similar in math and differ in API, habitual ecosystems, and paths to production. The choice answers “what task, what hardware, who maintains it,” not a hunt for an absolute champion. Transformers, YOLO, and other libraries live on top of the framework; JAX and teaching stacks micrograd/tinygrad evolve alongside. A logical path through the series:

How neural nets are trained → ML frameworks → PyTorch vs TensorFlow
        → YOLO / CRNN / Transformers → CV / LLM
        → ONNX → WebGPU / browser inference