AI Interpretability School — Explainable AI course, from SHAP to sparse autoencoders

AI Interpretability School — an open course on Explainable AI Interpretability is taught as a pile of scattered tutorials: one on SHAP, one on sparse autoencoders, nothing showing they are the same subject. AI Interpretability School is the missing path through it — free, in English and Russian, by Sabrina Sadiekh and Elena Ericheva. Track A, XAI Practitioner: SHAP, LIME, permutation importance, PDP/ICE/ALE, counterfactuals, feature interaction, Grad-CAM, SmoothGrad, Integrated Gradients, LRP, TCAV. Track B, Math of LLMs: the residual stream and QKV geometry, MLPs as key-value memory, superposition, circuits, activation patching, steering, sparse autoencoders, Gemma Scope. Track A — XAI Practitioner

Introduction — What the course is, how it works and what to know beforehand. Plus an entry test on machine learning.

XAI: motivation, introduction, terminology — Why explainability is needed, what regulation demands, and how explanation methods are classified.

Linear interpretable models — Linear and logistic regression, regularization, GLM/GAM/EBM.

Causality and Linear Models — When a coefficient reads as an intervention effect: instrumental variables and double machine learning.

Nonlinear interpretable models — Trees, forests, boosting importances, prototype networks.

Heterogeneous effects: whom the intervention actually helps — Whom an intervention helps and whom it does not: meta-learners, causal forest, honest splitting.

Explanation Methods for Machine Learning Models pt. 1 — How a feature moves a prediction: CPO, ICE, PDP and ALE.

Explanation Methods for Machine Learning Models pt. 2 — Permutation importance, the SHAP family, LIME, Anchors and counterfactual explanations — each with practice.

Feature interactions — Interaction maps, SHAP for interactions, Friedman's H-statistic, and which of them to trust.

What the network has learned: features and concepts — Activation map visualisation, feature visualization, network dissection, and the question of one unit per concept.

Attribution: the gradient as a heuristic — Saliency and SmoothGrad: what the gradient shows and where it misleads.

Attribution: localisation and the CAM family — CAM, Grad-CAM and their successors: attribution tied to a place in the image.

Attribution from axioms: IG, DeepLIFT, LRP — Integrated Gradients, DeepLIFT and LRP — methods derived from requirements rather than intuition.

Explanation Methods in DL pt. 4: Attention-based Explainability — Attention in transformers, rollout, flow, attention×gradient.

LLM Analysis — Embedding analysis, probing, prompting as XAI.

Explaining Agent Behaviour — Reading agent traces: which tool, and why the model reasoned it needed one.

Evaluating the Quality of Explanations — Co-12, sanity checks, ROAR and ROAD: how to tell whether an explanation is true.

Explaining DL Models via Perturbation — Ablation, occlusion, permutation and RISE: explaining through what breaks under intervention.

Attacks on Explanations — The thesis, a taxonomy of threats along four axes, and three failure modes of an explanation.

Track B — Math of LLMs

Explanation Methods in DL pt. 3: Concept-based Methods — XRAI, T-CAV, Concept Bottleneck and Concept Embedding models — explanations in concepts rather than pixels.

Methods for Analyzing Internal Representations — PCA, autoencoders and sparse autoencoders, the logit lens — how to read what happens inside a model.

Create a free account