Long-Context Fine-Tuning with Limited VRAM (Vladimir Fedosov et al., arXiv:2607.15105), posted to arXiv on July 16, pulls a long-standing track — "supervised fine-tuning of long-context LLMs" — explicitly down to the 16GB single-card tier for the first time, and validates the length can be pushed all the way to 131,072 tokens. In the agent / coding era this is an under-appreciated point, but the moment the engineering threshold is broken, many teams' iteration pace will be redefined. It brings together three pieces in one go: (1) Hierarchical Global Attention (HGA), slicing the long sequence into segments, each query block loading a bounded set of exact historical tokens, replacing dense attention's O(L²); (2) segment-wise backpropagation, only the active segment's computation graph stays differentiable in memory, old segments' KV explicitly detached; (3) tiered KV storage, old KV sinks through VRAM → RAM → NVMe in three tiers, letting "training" and "inference" share the same context storage. The numbers are tough: Qwen3-8B + 4-bit QLoRA + PG19, on the same 16GB Quadro RTX 5000, dense attention hits the ceiling at 2,048 tokens and goes straight to OOM at 4,096; HGA runs to 16,384 tokens with only 15.28GB peak memory; at the same time, the same adapter evaluated sequentially goes all the way down to 131,072 tokens, the only bottleneck being RAM/NVMe. At 2K training length for an apples-to-apples dense-attention readout, HGA-trained gets 2.7405 nat, dense-trained gets 2.7383 nat, the stock model is 2.9541 nat — that is, the model quality is almost aligned, while HGA is already 217.75 vs 207.02 tokens/s slightly faster, and the advantage grows as the context lengthens. The strategic meaning of this line worth discussing: over the past year the training threshold for long-context LLMs has shut many small and mid-sized labs out; cache-compression and hybrid-attention routes mostly only optimized the inference side, while the real training side has remained stuck on multi-card + high-bandwidth memory. HGA's path makes the training-side memory access pattern into "segment-locality + tiered storage" — very much like demand paging on CPUs, in some sense it's the GPU-side LPU/swap-out. Once this path continues to scale up along the agent training data size, "training million-token context on a consumer-grade card" turns from a slogan into a paper-worthy engineering reality, and the open-source ecosystem will pull back to the same starting line as closed-source APIs in the long-context scenario.