ML Research Engineer — LLM Interpretability & Quantization Geometry An interpretability research project at the frontier of a fast-moving field. Quantization lets open-weight LLMs run cheaper and faster, but it deforms their internal representations. Existing tools measure how much those representations change; this project measures the direction in which they break, and tests whether that direction-specific distortion predicts downstream model failure better than current metrics. The outcome is valuable either way a new diagnostic if the metric works and it feeds directly into a tool already in development. The work. Extract paired activations from Llama 3.1 8B and its quantized variants. Compute a per-layer, SVD-based geometric metric. Reproduce a recent baseline paper (PRISM, 2026) from its public repository. Run a statistical analysis testing whether the new metric adds signal beyond the baseline. The evaluation set is provided by us; the metric itself runs on a standard public text corpus (e.g. WikiText / C4) that you can pull directly. You build the pipeline, not the datasets. The engagement. Two paid phases: • Trial — implement the synthetic-data test suite and the SVD-based metric module, and reproduce one baseline axis on a small activation sample we provide. Deliverable: working, tested estimation code that passes the specified correctness checks. The full implementation phase is contingent on a successful trial. • Full implementation — the complete pipeline end to end: activation extraction, the full metric across layers, the PRISM reproduction plus secondary baselines, and the statistical signal analysis. Deliverable: an installable, documented Python package (automated tests, CLI) together with the computed results — per-layer distortion profiles, baseline comparisons, and the statistical analysis with plots — and a short report stating whether the metric adds signal. The fit. Strong Python and PyTorch (forward hooks), NumPy/SciPy, and applied linear algebra (SVD, PCA, ridge). Comfortable reading a paper's math and reproducing it against a public repo. Statistical literacy for regression and partial correlations. Bonus: experience with open-weight LLMs, quantization (GGUF/AWQ/GPTQ), or interpretability tooling (nnsight, TransformerLens, baukit). You'll need access to a GPU with ~24GB VRAM (e.g. an RTX 3090/4090) to run the 8B model locally.
Years: 2+
Location: Anywhere
Requested on: 2026-07-09
Machine learning, Tensorflow, PyTorch