DeepSeek V4 hybrid attention: 1M context at 27% of the compute

MorphLLM provides a deep analysis of DeepSeek V4's hybrid attention architecture. The combination of sliding window attention, full attention, and a "mHC" (multi-head compression) mechanism achieves the same quality as full attention at 1M context, with only 27% of the compute.