Tag: #reproducibility
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 4 posts
Reading a Technical Report Critically — What Gets Written Down and What Goes Missing
How to separate the verifiable from the unverifiable in a model technical report. Covers the limits of self-reported benchmarks, the places where a config and a report disagree, values that could not be read from gated r
2026-08-12 · 7 min read #ai-papers#model-internals#tech-report#benchmarks#evaluationWhat LLM Ops Actually Does — Reproducibility, Contamination, Checkpoints, Promotion, and Rollback
This post organizes LLM Ops as a list of responsibilities, not a list of tools. It covers what belongs in a run manifest that lets you reconstruct a training run, how to prevent and audit eval-set contamination, the form
2026-08-02 · 14 min read #mlops#llmops#reproducibility#evaluation#model-registryTaking Apart LLM Benchmark Tooling — Why the Same MMLU Gives Different Scores in Different Harnesses
A benchmark score is not a property of the model — it is the result of a measurement performed under specific conditions. The incident where the same LLaMA 65B scored 63.6 and 48.8 on the same MMLU at the same time shows
2026-08-02 · 21 min read #llm-evaluation#benchmark#lm-evaluation-harness#helm#reproducibilityThe New Python Notebook Stack in 2026 — Marimo, Quarto, and the Post-Jupyter Data Workflow (Observable, Pluto, Polars, DuckDB)
Jupyter's hidden state, non-diffable JSON, and reproducibility problems are still the biggest friction in 2026 data-science workflows. Marimo stores notebooks as .py files and executes them via a dataflow graph (not cell
2026-05-14 · 18 min read #marimo#quarto#jupyter#python-notebook#reactive-notebook