Math + LaTeX/KaTeX Complete Guide for Reading AI/ML Papers

Introduction

Open an AI/ML paper for the first time and the thing that feels most like a wall is the formulas. A developer who can read and write code can usually follow the algorithm itself, but without knowing the notation you get stuck on page one.

This article pulls together, in one place, "the minimum math you need in order to read papers" and "the LaTeX/KaTeX syntax that expresses that math". The goal is not to study a textbook cover to cover — it is to pick up quickly the patterns that show up over and over in real papers.

1. A Math Roadmap for Papers

1.1 Linear Algebra

Vectors and Matrices

In papers, data is almost always represented as a vector or a matrix.

• Scalar: $x \in \mathbb{R}$ — a single real value
• Vector: $\mathbf{x} \in \mathbb{R}^n$ — an $n$-dimensional column vector
• Matrix: $\mathbf{W} \in \mathbb{R}^{m \times n}$ — $m$ rows, $n$ columns

% Vector (boldface)
\mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}

% Matrix
\mathbf{W} = \begin{bmatrix}
  w_{11} & w_{12} & \cdots & w_{1n} \\
  w_{21} & w_{22} & \cdots & w_{2n} \\
  \vdots & \vdots & \ddots & \vdots \\
  w_{m1} & w_{m2} & \cdots & w_{mn}
\end{bmatrix}

Core operation:

That single line is one layer of a neural network. $\mathbf{W}$ is the weights, $\mathbf{b}$ the bias.

Dot Product

The most basic operation for measuring the similarity of two vectors.

\mathbf{a}^\top \mathbf{b} = \sum_{i=1}^{n} a_i b_i

Eigendecomposition

For a matrix $\mathbf{A}$, this means finding the $\lambda$ (eigenvalue) and $\mathbf{v}$ (eigenvector) that satisfy $\mathbf{A}\mathbf{v} = \lambda \mathbf{v}$. It shows up as a core ingredient in PCA, spectral clustering, and similar methods.

Singular Value Decomposition (SVD)

Decomposes an arbitrary matrix $\mathbf{A} \in \mathbb{R}^{m \times n}$ into a product of three matrices.

• $\mathbf{U} \in \mathbb{R}^{m \times m}$: left singular vectors (orthogonal matrix)
• $\boldsymbol{\Sigma} \in \mathbb{R}^{m \times n}$: diagonal matrix of singular values
• $\mathbf{V} \in \mathbb{R}^{n \times n}$: right singular vectors (orthogonal matrix)

It turns up constantly in dimensionality reduction, recommender systems, LoRA, and more.

\mathbf{A} = \mathbf{U} \boldsymbol{\Sigma} \mathbf{V}^\top

Norms

Measure the "size" of a vector.

1.2 Calculus

Partial Derivatives

Differentiating a multivariable function with respect to one variable only.

It tells you how much each parameter of a network influences the loss.

Gradient (the vector of partial derivatives)

All the partial derivatives collected into a vector.

\nabla_{\boldsymbol{\theta}} \mathcal{L} = \begin{bmatrix}
  \frac{\partial \mathcal{L}}{\partial \theta_1} \\
  \frac{\partial \mathcal{L}}{\partial \theta_2} \\
  \vdots \\
  \frac{\partial \mathcal{L}}{\partial \theta_n}
\end{bmatrix}

Chain Rule

The differentiation rule for composite functions, and the mathematical basis of backpropagation.

A 3-layer network example:

Jacobian and Hessian

• Jacobian $\mathbf{J} \in \mathbb{R}^{m \times n}$: the first-derivative matrix of a vector function. $J_{ij} = \frac{\partial f_i}{\partial x_j}$
• Hessian $\mathbf{H} \in \mathbb{R}^{n \times n}$: the second-derivative matrix of a scalar function. $H_{ij} = \frac{\partial^2 f}{\partial x_i \partial x_j}$

The Hessian carries curvature information, so it is used in second-order optimization (Newton's method).

\mathbf{J} = \begin{bmatrix}
  \frac{\partial f_1}{\partial x_1} & \cdots & \frac{\partial f_1}{\partial x_n} \\
  \vdots & \ddots & \vdots \\
  \frac{\partial f_m}{\partial x_1} & \cdots & \frac{\partial f_m}{\partial x_n}
\end{bmatrix}

1.3 Probability and Statistics

Probability Distributions

Expectation and Variance

\mathbb{E}[X] = \sum_x x \, P(x)
\text{Var}(X) = \mathbb{E}[X^2] - (\mathbb{E}[X])^2

Conditional Probability and Bayes' Theorem

• $P(A)$: the prior
• $P(B \mid A)$: the likelihood
• $P(A \mid B)$: the posterior
• $P(B)$: the evidence (marginal likelihood)

MLE and MAP

Maximum Likelihood Estimation (MLE):

Maximum A Posteriori (MAP):

MAP is MLE with a prior distribution $p(\boldsymbol{\theta})$ added. L2 regularization is equivalent to MAP under a Gaussian prior.

1.4 Optimization

Gradient Descent

$\eta$ is the learning rate. Use mini-batches and it becomes SGD.

Adam Optimizer

The most widely used optimizer. It uses moving averages of the first moment (mean) and the second moment (variance).

\begin{aligned}
m_t &= \beta_1 m_{t-1} + (1-\beta_1) g_t \\
v_t &= \beta_2 v_{t-1} + (1-\beta_2) g_t^2 \\
\hat{m}_t &= \frac{m_t}{1-\beta_1^t}, \quad
\hat{v}_t &= \frac{v_t}{1-\beta_2^t} \\
\theta_{t+1} &= \theta_t
  - \frac{\eta}{\sqrt{\hat{v}_t} + \epsilon} \hat{m}_t
\end{aligned}

Regularization

Adds a penalty term to the loss function to prevent overfitting.

• $\lambda$: regularization strength (a hyperparameter)
• L1 regularization ($\lVert \boldsymbol{\theta} \rVert_1$): induces sparsity → feature selection
• L2 regularization ($\lVert \boldsymbol{\theta} \rVert_2^2$): weight decay → bounds parameter magnitude

Convex vs Non-Convex

• Convex: the global minimum equals the local minimum. Logistic regression, SVMs, and so on.
• Non-convex: many local minima. Most deep-learning loss functions fall here.

When a paper says "non-convex optimization", it means there is no guarantee of reaching the global optimum.

2. Formula Patterns That Recur in Papers

2.1 Softmax and Cross-Entropy

Softmax: converts a logit vector into a probability distribution.

Cross-Entropy Loss: the standard loss function for classification.

Here $y_i$ is the one-hot label and $\hat{y}_i$ is the softmax output.

\text{softmax}(z_i) = \frac{e^{z_i}}{\sum_{j=1}^{K} e^{z_j}}

\mathcal{L}_{\text{CE}} = -\sum_{i=1}^{K} y_i \log(\hat{y}_i)

2.2 Attention Score and Scaled Dot-Product

Scaled Dot-Product Attention, the heart of the Transformer:

• $\mathbf{Q} \in \mathbb{R}^{n \times d_k}$: the Query matrix
• $\mathbf{K} \in \mathbb{R}^{m \times d_k}$: the Key matrix
• $\mathbf{V} \in \mathbb{R}^{m \times d_v}$: the Value matrix
• $\sqrt{d_k}$: the scaling factor (keeps the dot products from growing large)

Multi-Head Attention:

\text{Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{V})
  = \text{softmax}\!\left(
    \frac{\mathbf{Q}\mathbf{K}^\top}{\sqrt{d_k}}
  \right)\mathbf{V}

2.3 KL Divergence and ELBO

KL Divergence: measures the "distance" (asymmetric) between two probability distributions.

Properties:

• $D_{\text{KL}} \geq 0$ (always 0 or greater)
• $D_{\text{KL}}(q \| p) \neq D_{\text{KL}}(p \| q)$ (asymmetric)

ELBO (Evidence Lower Bound): the core objective function of the VAE.

• First term: reconstruction loss — how well the decoder restores the input
• Second term: the KL term — how close the encoder's posterior is to the prior

\log p(\mathbf{x}) \geq
  \underbrace{\mathbb{E}_{q(\mathbf{z}|\mathbf{x})}
  [\log p(\mathbf{x}|\mathbf{z})]}_{\text{reconstruction}}
  - \underbrace{D_{\text{KL}}(q(\mathbf{z}|\mathbf{x})
  \,\|\, p(\mathbf{z}))}_{\text{regularization}}

2.4 Normalization and Loss Notation

Layer Normalization:

Here $\mu = \frac{1}{d}\sum_{i=1}^d x_i$ and $\sigma^2 = \frac{1}{d}\sum_{i=1}^d (x_i - \mu)^2$, while $\boldsymbol{\gamma}$ and $\boldsymbol{\beta}$ are learnable parameters and $\odot$ is element-wise multiplication.

Batch Normalization:

$\mu_B$ and $\sigma_B^2$ are computed per mini-batch.

RMSNorm (used in LLaMA and others):

Loss notation conventions:

• $\mathcal{L}$ or $L$: the loss function as a whole
• $\mathcal{L}_{\text{CE}}$, $\mathcal{L}_{\text{MSE}}$: subscript notation for a specific loss
• $\ell(\cdot)$: per-sample loss (lowercase)
• $J(\boldsymbol{\theta})$: the objective function (= loss + regularization)

3. The Essentials of Reading and Writing LaTeX

3.1 Basic Formula Syntax

Fractions, Sums, Integrals

% Fraction
\frac{a}{b}            % → a/b
\dfrac{a}{b}           % → large fraction (display style)

% Sum
\sum_{i=1}^{N} x_i     % → Σ from i=1 to N
\prod_{j=1}^{M} p_j    % → Π from j=1 to M

% Integral
\int_{a}^{b} f(x)\,dx           % → definite integral
\iint_D f(x,y)\,dx\,dy          % → double integral
\oint_C \mathbf{F}\cdot d\mathbf{r}  % → line integral

Rendered result:

Subscripts and Superscripts

x_i           % subscript
x^2           % superscript (square)
x_i^{(j)}    % combined: the j-th iteration of the i-th element
\hat{y}       % y-hat (prediction)
\bar{x}       % x-bar (mean)
\tilde{x}     % x-tilde (transformed value)
\dot{x}       % time derivative
\mathbf{W}    % bold (matrix/vector)
\mathcal{L}   % calligraphic (loss function)
\mathbb{R}    % blackboard bold (number systems)

Patterns you see very often in papers:

Matrices

% Parenthesis matrix
\begin{pmatrix} a & b \\ c & d \end{pmatrix}

% Bracket matrix
\begin{bmatrix} a & b \\ c & d \end{bmatrix}

% Determinant
\begin{vmatrix} a & b \\ c & d \end{vmatrix}

3.2 Alignment Environments

aligned (aligning multi-line formulas)

$$
\begin{aligned}
  f(x) &= ax^2 + bx + c \\
       &= a(x - h)^2 + k \\
       &= a(x - r_1)(x - r_2)
\end{aligned}
$$

The & symbol sets the alignment point, and \\ breaks the line.

cases (conditional branches)

$$
f(x) = \begin{cases}
  1 & \text{if } x > 0 \\
  0 & \text{if } x = 0 \\
  -1 & \text{if } x < 0
\end{cases}
$$

You see it often in definitions of activation functions such as ReLU:

3.3 How to Read the Macros Papers Commonly Use

You rarely look at LaTeX source directly from a paper PDF, but when you pull the arXiv source or write a blog post, certain macros come up again and again.

% Common user-defined macros
\newcommand{\E}{\mathbb{E}}           % expectation
\newcommand{\R}{\mathbb{R}}           % the reals
\newcommand{\KL}{D_{\text{KL}}}       % KL divergence
\newcommand{\norm}[1]{\lVert #1 \rVert}  % norm
\newcommand{\inner}[2]{\langle #1, #2 \rangle}  % inner product

% Usage examples
\E_{x \sim p}[f(x)]     % → 𝔼_{x~p}[f(x)]
\norm{\mathbf{x}}_2     % → ‖x‖₂
\inner{\mathbf{a}}{\mathbf{b}}  % → ⟨a, b⟩

Tip: to see the source of an arXiv paper, change abs to e-prints in the URL, or click "Download source" to get the .tex file. Checking the \newcommand definitions first makes decoding the formulas in the body far easier.

4. KaTeX in Practice

4.1 Rendering KaTeX in an MDX Blog

To use KaTeX on a Next.js + Contentlayer + MDX stack like this blog, you need two plugins.

npm install remark-math rehype-katex

An example Contentlayer configuration:

// contentlayer.config.ts
import remarkMath from 'remark-math'
import rehypeKatex from 'rehype-katex'

export default makeSource({
  mdx: {
    remarkPlugins: [remarkMath],
    rehypePlugins: [rehypeKatex],
  },
})

Then add the KaTeX CSS to your layout:

<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/katex@0.16.8/dist/katex.min.css" />

4.2 When to Use Inline vs Block Formulas

Inline formulas ($...$): when the math flows naturally inside a sentence.

Differentiating the loss function $\mathcal{L}$ with respect to $\boldsymbol{\theta}$ gives...

→ Differentiating the loss function $\mathcal{L}$ with respect to $\boldsymbol{\theta}$ gives...

Block formulas ($$...$$): when a standalone formula deserves emphasis.

$$
\nabla_{\boldsymbol{\theta}} \mathcal{L} = \frac{1}{N} \sum_{i=1}^{N} \nabla_{\boldsymbol{\theta}} \ell(f(\mathbf{x}_i; \boldsymbol{\theta}), y_i)
$$

How to decide:

• 1–2 symbols → inline
• Tall expressions such as fractions, sums, matrices → block
• The central formula of an explanation → block
• A variable mentioned for reference → inline

4.3 Common Rendering Errors and Fixes

1) Special-character errors inside a \text{} block

% Error
\text{cross-entropy}    % the hyphen may be read as a minus sign

% Fix
\text{cross\text{-}entropy}
% or simply drop the hyphen
\text{cross entropy}

2) \\ line breaks do not work

In KaTeX, \\ works only inside environments such as aligned, cases, and bmatrix. To break a line in a plain block formula, use a separate $$ block or wrap it in aligned.

% Error: breaking a line directly in a block formula
$$
a = b \\
c = d
$$

% Fix: use aligned
$$
\begin{aligned}
a &= b \\
c &= d
\end{aligned}
$$

3) \boldsymbol vs. \mathbf: which to use

• \mathbf{x}: upright bold. Latin alphabet only.
• \boldsymbol{\theta}: italic bold. Also works for Greek letters.

\mathbf{W}              % matrix W (upright bold) ✅
\mathbf{\theta}         % ❌ not recommended for Greek letters
\boldsymbol{\theta}     % ✅ bold Greek letters

4) A $$ block needs blank lines around it

This is the most common reason a $$ block formula fails to render in MDX.

<!-- Error: no blank lines -->

This is a formula:

$$
x = y
$$

The next sentence.

<!-- Fix: add blank lines before and after -->

This is a formula:

$$
x = y
$$

The next sentence.

5) Commands KaTeX does not support

KaTeX supports only a subset of LaTeX. Unsupported commands you hit often:

5. How to Practise in Real Life

5.1 A Checklist for Reading a Single Paper

1. Abstract → grasp the purpose: state in one sentence the problem the paper is trying to solve
2. Skim figures and tables first: Figure 1 is usually a summary of the whole architecture
3. Find the notation table: normally laid out on page 2 or 3
4. Identify the core formulas: usually 3 to 5. Focus on the numbered equations
5. Translate the formulas into plain language: unpack each one as an "input → operation → output" flow
6. Check the experimental results: read the improvement over the baseline off the tables and graphs
7. Interpret the ablation study: work out which component contributes to the performance
8. Look at the code: cross-check the published GitHub implementation against the formulas

5.2 How to Back-Trace When a Formula Stops You

Build a symbol dictionary

When you meet a symbol you do not know while reading, write it into your notes immediately.

## Symbol dictionary (Paper: Attention Is All You Need)

| Symbol                               | Meaning                    | Dimension                                  | Notes                  |
| ------------------------------------ | -------------------------- | ------------------------------------------ | ---------------------- |
| $d_{\text{model}}$                   | Model hidden dimension     | scalar                                     | 512                    |
| $d_k$                                | Key/Query dimension        | scalar                                     | $d_{\text{model}} / h$ |
| $d_v$                                | Value dimension            | scalar                                     | $d_{\text{model}} / h$ |
| $h$                                  | Number of heads            | scalar                                     | 8                      |
| $\mathbf{Q}, \mathbf{K}, \mathbf{V}$ | Query, Key, Value          | matrix                                     |                        |
| $\mathbf{W}^Q_i$                     | Query projection of head i | $\mathbb{R}^{d_{\text{model}} \times d_k}$ |                        |

Back-tracing in 3 steps:

1. Find where the symbol is defined: search the text above and below the formula for "where" or "here"
2. Back out the dimensions: for a matrix product to be valid the inner dimensions must agree — use that to infer the dimension of an unknown symbol
3. Cross-check the code: confirm what the formula means from code such as attention_scores = torch.matmul(Q, K.transpose(-2, -1)) / math.sqrt(d_k)

5.3 Example: Scaled Dot-Product Attention Worked Step by Step

Step 1: check the inputs

• $\mathbf{Q} \in \mathbb{R}^{n \times d_k}$: $n$ queries, each $d_k$-dimensional
• $\mathbf{K} \in \mathbb{R}^{m \times d_k}$: $m$ keys, each $d_k$-dimensional
• $\mathbf{V} \in \mathbb{R}^{m \times d_v}$: $m$ values, each $d_v$-dimensional

Step 2: compute the $\mathbf{Q}\mathbf{K}^\top$ product

Element $(i, j)$ is the dot product of the $i$-th query and the $j$-th key → a similarity score

Step 3: divide by the $\sqrt{d_k}$ factor

The larger $d_k$ is, the larger the dot products get in absolute value. Dividing by $\sqrt{d_k}$ keeps softmax from saturating. That is what "scaled" means.

Step 4: apply softmax

Apply softmax over each row (each query) to produce the attention weights (a probability distribution).

Step 5: take the weighted sum over Value

Sum the Value vectors, weighted by the attention weights.

Result: $\text{output} \in \mathbb{R}^{n \times d_v}$

One-line summary: "each query measures its similarity to every key, then uses those similarities as weights to output a weighted average of the values."

The matching PyTorch code:

import torch
import torch.nn.functional as F
import math

def scaled_dot_product_attention(Q, K, V):
    d_k = Q.size(-1)
    scores = torch.matmul(Q, K.transpose(-2, -1)) / math.sqrt(d_k)
    attn_weights = F.softmax(scores, dim=-1)
    output = torch.matmul(attn_weights, V)
    return output, attn_weights

6. Appendix

6.1 Symbol Cheat Sheet

6.2 Recommended Resources

Textbooks

1. Mathematics for Machine Learning (Deisenroth et al.) — Free PDF available. Focuses only on the math ML actually needs
2. Deep Learning (Goodfellow, Bengio, Courville) — Part I is the math foundation. Free to read at deeplearningbook.org
3. Pattern Recognition and Machine Learning (Bishop) — Reaches as far as probabilistic graphical models. A textbook from the Bayesian viewpoint
4. Linear Algebra Done Right (Axler) — Theory-centred linear algebra. Proof-based, suited to a deep understanding

Online courses

5. 3Blue1Brown — Essence of Linear Algebra (YouTube) — A visualization-driven series that builds linear-algebra intuition
6. 3Blue1Brown — Essence of Calculus (YouTube) — Visual understanding of the core calculus concepts
7. Stanford CS229 — Machine Learning — Andrew Ng's ML course. Mathematically solid
8. MIT 18.06 — Linear Algebra (Gilbert Strang) — A celebrated linear-algebra course from an engineering perspective

Tools and references

9. KaTeX official docs (katex.org) — The complete list of supported functions and symbols
10. Detexify (detexify.kirelabs.org) — Draw a symbol by hand and it finds the LaTeX command for you
11. Mathpix Snip — An OCR tool that turns a photo of a formula into LaTeX code
12. Overleaf (overleaf.com) — An online LaTeX editor with live preview
13. arXiv Vanity — Converts arXiv papers into web pages so the formulas are comfortable to read
14. Papers with Code (paperswithcode.com) — Papers and their implementations side by side. Ideal for matching formulas against code

Conclusion

A paper's formulas are, in the end, a compressed expression of "how the data gets transformed". Every symbol feels unfamiliar at first, but the patterns collected above cover more than 80% of the formulas you will meet across papers.

The most effective learning order is:

1. Keep this article's symbol cheat sheet beside you
2. Pick one paper you are interested in
3. Write out just the 3 to 5 core formulas in plain language
4. Cross-check them against the matching code (PyTorch/JAX)

Once you can translate formulas into code and code back into formulas, your reading speed goes up dramatically.