Engineering Deep Dives

The Problem

Memory leaks in LLMs are hard to catch because PyTorch’s caching allocator acts as a black box. Standard tools like nvidia-smi only show total GPU memory, completely missing internal allocator fragmentation and unreleased tensor references post-generation.

The Approach

Built KVScope, a profiling framework that injects forward hooks into attention modules. By intercepting DynamicCache objects, it accurately captures KV cache tensor snapshots at every step without requiring changes to application code, using six heuristic detectors for specific memory pathologies.

Key Findings

• Gemma 4 post-EOS retention: 4.7–5.3 GB retained per generation
• gpt-oss-120B allocator gap: 14.5 GiB reserved-but-unused on H100

The Problem

A slow memory leak takes hours or days to kill a container, but a release pipeline stage runs for minutes. Most monitoring only alerts once memory is already exhausted. The question was whether a leak can be caught reliably from a short observation window, before release, without drowning teams in false alarms.

The Approach

eBPF and container-stats collectors sampling at 1 Hz fed TimescaleDB from workloads on four cloud providers (Oracle, IBM, Google Cloud, DigitalOcean). I derived how the smallest detectable leak rate scales with the observation window to bound the design, then compared ten detectors, from change-point methods to a gradient-boosted classifier, all held to the same 5% false-alarm budget on healthy data. A CI gate blocks a release when a leak is detected.

Key Findings

• 703,000 records over 755 runs, audited: 363 runs whose sampling had silently drifted below target were handled explicitly, not averaged away
• Held to one 5% false-alarm budget, six of the ten detectors lost most of their recall
• Primary detector (gradient-boosted trees): 98.6% recall at a 4.9% false-alarm rate
• An LSTM autoencoder scored below chance, so I retired it and reported it as a negative result
• Supervised by Ben Carpenter, EPCC

The Problem

Forecasting where lightning will strike means combining numerical weather model output with what ground sensors actually record. The internship asked whether a spatio-temporal deep learning model could learn that mapping.

The Approach

Trained a ConvLSTM encoder-decoder in PyTorch on WRF numerical weather output (CAPE, vertical velocity, cloud ice water content) fused with NRSC ground-sensor records of more than 163,000 strike events, and compared three architectures. Built a React and Leaflet dashboard with state-level filtering.

Outcome

• Submitted as a research prototype to ISRO's internal research library
• Three architectures compared on the same fused dataset
• My first work on an HPC system