Archive

2026

• July 20th — Power by the hour (Economics, ML Infrastructure) (ml, post-training)
• July 11th — Who is walking who? (Economics) (big-tech, llm, post-training)
• July 9th — MOPD (links-and-recs)
• June 30th — Benchmarks Mean Business (Economics) (big-tech, llm)
• June 28th — It’s always the learning rates (Modelling) (training)
• June 19th — LLMs are complicated now (Modelling) (gpu, llm, recsys, triton)
• June 12th — FactWorld (Modelling) (llm, ml, training)
• June 5th — Somehow, more on distillation (Modelling, posts) (training)
• May 31st — We can distill it for you wholesale (Modelling) (post-training)
• May 27th — Maybe the agents shouldn’t write the kernels (ML Infrastructure) (gpu, triton)
• May 25th — The elusive order of things (ML Infrastructure, posts) (gpu, triton)
• April 27th — Loss Exploded. (Modelling) (big-tech, llm, training)
• April 8th — Unbundling Work (Economics) (big-tech)
• March 17th — Native DSLs Ops in PyTorch (til) (pytorch, triton)
• March 10th — Programming The Loop (Economics) (agents, big-tech)
• March 6th — Cutie Fly (ML Infrastructure) (cute, ml)
• February 24th — Perplexed (links-and-recs) (training)
• February 22nd — The model can probably write the code (Economics) (big-tech, devx, llm)
• February 11th — TileIR (ML Infrastructure) (gpu, triton)
• February 9th — What is In-Distribution (links-and-recs) (post-training, training)
• February 2nd — You can just do things, but you don’t have to (Psychology) (big-tech)
• January 28th — Do MoEs Think Different? (links-and-recs) (llm, moe)
• January 27th — Anyone got any Veras? (links-and-recs)
• January 20th — Everything MoE (Modelling) (ml, training)
• January 13th — Quis Custodiet Ipsos Custodes (Economics) (big-tech)
• January 1st — Megacores (ML Infrastructure) (gpu, ml)

2025

• December 21st — Comparative Human Advantage (Economics) (big-tech, ml)
• December 5th — Attention, Compression & Predicting the next token (Modelling) (llm, training)
• November 9th — Bulls in the bazaar (Economics) (big-tech, oss)
• November 3rd — Let’s all switch to FP16? (ML Infrastructure) (gpu, llm, post-training)
• October 22nd — Helion and the evolving GPU programming model (ML Infrastructure) (gpu, pytorch)
• September 29th — Qwen-Image (Modelling) (diffusion, training)
• September 3rd — Automation & Managerial Control (Economics) (big-tech)
• September 2nd — A Primer on Post-Training (links-and-recs) (post-training)
• August 26th — Layouts (links-and-recs) (gpu, pytorch)
• August 19th — The TPU book, on GPUs (links-and-recs) (gpu, training)
• August 13th — Extending Arcee’s FM context length (links-and-recs) (post-training)
• August 12th — Rubrics (Modelling, note to self) (llm, post-training)
• August 6th — Constraints & Orchestrators (ML Infrastructure) (ml, python)
• August 4th — Overthinking Everything (Modelling) (llm, training)
• July 30th — The Tools Are Made Up (Modelling) (ml, post-training, training)
• July 25th — PyTorch Conference 2025 (ML Infrastructure) (pytorch)
• July 22nd — RL in the second half (links-and-recs) (ml, post-training)
• July 18th — Quack CuteDSL Kernels (links-and-recs) (gpu)
• July 9th — Reinforcement Learning Continues To Be The Frontier (note to self) (ml, post-training)
• July 8th — ARPA and predicting the future (links-and-recs) (big-tech)
• July 4th — Cute-DSL (ML Infrastructure) (gpu, python)
• June 28th — Harnessing the Universal Geometry of Embeddings (links-and-recs) (ml)
• June 27th — How to build unmaintainable kernels (ML Infrastructure) (gpu, ml)
• June 26th — GPU Driven (note to self) (gpu)
• June 21st — AbsenceBench (links-and-recs) (ml, post-training)
• June 21st — Free-Threaded Python gets ‘supported’ status (ML Infrastructure) (python)
• June 15th — A patchwork quilt view of AI Alignment (links-and-recs) (ml)
• June 11th — Linear Layouts in Triton (links-and-recs) (gpu, triton)
• June 10th — Monarch: PyTorch Single Controller (ML Infrastructure) (python, pytorch)
• June 10th — Toward a Theory of Tokenization in LLMs (links-and-recs) (llm, ml)
• June 5th — Analyzing Modern GPU Cores (links-and-recs) (gpu)
• June 1st — Darwin Gödel Machines (links-and-recs) (agentic, ml)
• May 30th — Keeping a GPU busy is a lot about tiling (note to self) (gpu)
• May 28th — Metrics for Engineering Teams (Economics) (big-tech)
• May 27th — Scaling RL Compute (links-and-recs) (post-training)
• May 24th — Fused Linear Cross-Entropy (ML Infrastructure) (llm, pytorch)
• May 23rd — Values in AI (Psychology) (big-tech)
• May 19th — Scientific discovery and AI (links-and-recs) (ml)
• May 17th — Optimizers and Hessians (links-and-recs) (ml)
• May 16th — Pyrefly (ML Infrastructure) (python)
• May 16th — The First Year of Free-Threaded Python (links-and-recs) (python)
• May 13th — Generative modelling in the latent space (links-and-recs) (ml)
• May 9th — How does Triton do Warp Spec? (note to self) (gpu, triton)
• May 8th — The Mold Linker (links-and-recs) (devinfra)
• May 7th — LSP & Standards (links-and-recs) (big-tech, python)
• May 6th — Scalably Solving Assistant Games (links-and-recs) (ml, post-training)
• May 5th — Richard Zou on torch.compile (links-and-recs) (pytorch)
• May 4th — Autotuning in PyTorch & Triton (note to self) (gpu, triton)
• May 1st — Profiling Triton (note to self) (gpu, triton)
• April 18th — JEPA (note to self) (ml)
• April 17th — Free-Threaded Python community (ML Infrastructure) (python)
• April 17th — Colfax on Blackwell GEMMs (links-and-recs) (gpu)
• April 12th — MegaScale-infer: disagg MoE inference (links-and-recs) (inference, llm)
• April 11th — Pydantic Evals (note to self) (ml)
• April 10th — Bonanza Build System (links-and-recs) (build)
• April 5th — Improving Recommendation Systems with LLMs (links-and-recs) (recsys)
• April 4th — Dynamic Shapes in PyTorch (note to self) (pytorch)
• April 3rd — PyTorch and Python Free Threading (links-and-recs) (python, pytorch)
• April 1st — Understanding R1-Zero-Like training (links-and-recs) (ml, post-training)
• March 29th — Bank Conflicts in Shared Memory (note to self) (gpu)
• March 28th — Useful Reasoning Behaviors (Psychology) (llm, post-training)
• March 27th — Scarcity & Abundance (links-and-recs) (big-tech, devinfra)
• March 23rd — Muon optimizer (note to self) (ml, training)
• March 21st — What if your worry problem is really a planning problem? (links-and-recs)
• March 20th — Libraries not Frameworks & Training vs Agentic loops (links-and-recs) (agentic, big-tech)
• March 19th — A HBS paper on the PyTorch Foundation (links-and-recs) (big-tech, pytorch)
• March 19th — Vibe Coding & Code Review (Economics) (big-tech)
• March 18th — Ping Pong GEMM from Scratch (links-and-recs) (gpu)
• March 16th — Post-Training & Elicitation (Modelling) (llm, post-training)
• March 14th — Quantization in PyTorch (ML Infrastructure) (pytorch)
• March 10th — Performance of the tail-call interpreter in Python 3.14 (links-and-recs) (python)
• March 4th — Horace He’s thoughts on PyTorch (links-and-recs) (big-tech, pytorch)
• February 26th — Write more kernels (links-and-recs) (gpu)
• February 25th — DeepSeek’s Flash MLA Kernel (links-and-recs) (gpu)
• February 23rd — Demystifying Long Chain-of-Thought Reasoning in LLMs (links-and-recs) (llm, post-training)
• February 18th — Byte-Latent Transformers (Modelling) (llm, ml, training)
• February 16th — Warp Specialization (note to self) (gpu)
• February 13th — Streaming DiLoCo (note to self) (llm, training)
• February 12th — Ping-Pong Kernels on Hopper (links-and-recs) (gpu, ml, pytorch)
• February 11th — Grouped GEMMs and MoE (note to self) (ml, training)
• February 8th — JIT decorators in Python (links-and-recs) (python)
• February 6th — On career growth (Economics) (big-tech)
• February 1st — Flow Matching (links-and-recs) (ml)
• February 1st — GRPO & Verifiable Rewards (Modelling) (llm, post-training, pytorch)
• January 26th — Gradient Accumulation (was) busted (ML Infrastructure) (llm, post-training, pytorch)
• January 25th — DeepSeek R1 & GRPO (note to self) (llm, post-training)
• January 21st — TIL: You don’t need PIL for decoding images with TorchVision (til) (pytorch)
• January 20th — Vita 1.5 – how to train an end-to-end speech model (links-and-recs) (llm, ml)
• January 16th — Functionalization in PyTorch (til) (pytorch)
• January 12th — Idle Speculation on GPU Capacity Management (note to self) (training)
• January 8th — TIL: weights-only model loading will be the default in PyTorch 2.6 (til) (pytorch)
• January 6th — Deepseek V3 (links-and-recs) (llm, ml)
• January 2nd — TIL: torchdbg (til) (pytorch)
• January 1st — HuatuoGPT-o1, domain specific reasoning (links-and-recs) (llm, ml, training)

2024

• December 31st — Thunderkittens /GPUs go brr (note to self) (gpu, ml)
• December 29th — Generalization progress in RLHF (links-and-recs) (llm, post-training)
• December 23rd — Ways to use torch.export (links-and-recs) (pytorch)
• December 22nd — TLParse (note to self)
• December 22nd — TIL: TunableOp in PyTorch (til) (pytorch)
• December 18th — nonzero_static in PyTorch (ML Infrastructure) (pytorch)
• December 17th — Diff Risk Score Model (links-and-recs) (llm, ml)
• December 15th — Chinese Tech Term Glossary (til) (big-tech)
• December 15th — ChatGPT Canvas can make API requests now, but it’s complicated (links-and-recs) (llm)
• December 13th — Things that can go wrong with priority levels (links-and-recs) (build)
• December 11th — Way to use torch.compile (links-and-recs) (pytorch)
• December 11th — Engineering Culture at Meta (Psychology) (big-tech)
• December 6th — A very lazy mental model for parallelisms (ML Infrastructure) (ml)
• December 5th — TIL: When does PyTorch upgrade Python versions (til)
• November 29th — A Meticulous Guide to Advances in Deep Learning Efficiency over the Years (links-and-recs) (ml)
• November 28th — Training Language Models to Self-Correct via Reinforcement Learning (links-and-recs) (ml)
• November 27th — PyTorch while_loop (ML Infrastructure) (pytorch)
• November 26th — Eureka moments (links-and-recs)
• November 25th — Unsupported: dynamic shape operator: aten.nonzero.default with boolean masks in torch.compile (til) (pytorch)
• November 20th — TIL: ROCM is actually open source (til)
• November 19th — Notes to Self: Torch Compile References (note to self)
• November 15th — TIL: conda cudatoolkit (til)
• November 5th — TIL: Numerical stability in TorchScript vs Export (til) (pytorch)
• November 2nd — TorchFT: Fault tolerant training (links-and-recs) (pytorch, training)
• September 24th — Alphafold3 PyTorch (links-and-recs) (pytorch)
• July 14th — Thinking about high quality human data (links-and-recs) (training)
• June 28th — What are NPUs (links-and-recs) (inference)
• June 24th — Daniel Schmachtenberger’s Questions (links-and-recs) (big-tech)
• June 24th — HSTU (links-and-recs) (recsys)
• May 24th — On what neural net architectures actually make a difference (links-and-recs) (ml)
• April 22nd — TIL: How to measure memory usage from your PyTorchmodel without running it (til) (pytorch)
• April 21st — Timeline on the Google Reader shutdown (til) (big-tech)
• February 22nd — DiLoCo (til)
• January 16th — Inductor notes (note to self) (pytorch)
• January 8th — PyTorch 2 Internals (links-and-recs) (pytorch)
• January 4th — Better performance on GPUs (links-and-recs) (gpu)

2023

• October 7th — How AMD may get across the CUDA moat (links-and-recs) (gpu, pytorch)
• September 19th — Pytorch composability meeting notes (note to self) (pytorch)
• June 29th — Transformer model inference optimizations (links-and-recs)
• June 28th — CUDA graphs in PyTorch (til) (pytorch)
• June 9th — PyTorch Internals (links-and-recs)
• April 10th — Ekko: Low Latency Model Updates at Tencent (links-and-recs) (recsys)
• March 31st — Sparks of AGI (links-and-recs)
• March 24th — Monolith: Real Time Recommendation System (links-and-recs) (recsys)
• March 20th — On The Factory Floor (links-and-recs) (recsys)
• February 24th — Rich Sutton’s Bitter Lesson (til)