Video generation post-training is moving toward reinforcement learning, but the reward stage has always been the hard part: code and math have a single correct answer that can be checked automatically, while whether a video "conveys the right emotion" has no standard answer. Fusing fixed weights for visual quality, motion quality, and text alignment tends to miss emotional intent and leaves room for reward hacking. RLHF can handle subjective goals, but annotation is expensive, and individual aesthetic differences make facial, gestural, and chromatic cues hard to calibrate across narrative contexts.

EmoReward reframes the question. Instead of asking which reward is more accurate, it asks whether a given piece of evidence should be trusted at all, and how much.

Two granularities: within one video, and between two training runs

The paper comes from HKUST, Tencent Hunyuan, Fudan University, and the University of Waterloo. It is public on arXiv (cs.CV / cs.AI, 30 pages, 15 figures), with a project page and an official Tencent Hunyuan repository.

The mechanism has two layers. Within a single video is Reflective Reward Gating: seven reward providers (a VLM judge, facial action units, body cues, temporal evidence, CLIP-AUTT, R1-Omni, and motion naturalness) score the video while keeping the criterion score s_k separate from whether that evidence is usable at all, u_k. Gates keyed on training phase, face confidence, semantic conflict, and subtlety decide how much weight each provider gets for that sample; unusable evidence is dropped from the active set and the weights are renormalized.

Between training runs is RewardMemory: after a run completes, provider reliability is estimated per emotional regime from the reward traces, converted into a bounded multiplier table, frozen, and reused in the next run. The multiplier is exp(Δ_k), where Δ_k comes from the provider's correlation with the reference outcome minus the mean correlation across providers, scaled and clipped to ±1. The key point is that no reward-model parameters are updated throughout; only the generator receives gradients.

The paper's formula is R_emo = Σ(w⁰·g·m)·s / Σ(w⁰·g·m) + ε, fused with a general video reward as R_total = (α·R_vid + β·R_emo)/(α+β).

Benchmarks: all three RL algorithms improve

The base model is HunyuanVideo 1.5, run with three RL algorithms — DanceGRPO, FlowGRPO, and SAGE-GRPO — at fusion weights α=β=0.5, batch size 2, and group size 4, evaluated on 302 emotional prompts that do not overlap the training set.

The VideoAlign Overall scores in Table 1: SAGE-GRPO rises from -0.8269 to -0.2624, FlowGRPO from -0.7719 to -0.5376, and DanceGRPO from -1.2812 to -0.9594. All four VideoAlign heads (Overall, visual quality, motion quality, text alignment) increase across all three algorithms. The DanceGRPO row uses Reflect rather than the complete EmoReward system, and the table labels the two configurations separately.

But the paper makes an important point itself: VideoAlign is also the optimization objective, so agreeing with it does not mean emotional fidelity improves independently. PickScore and CLIPScore, the two proxy metrics, move less and not uniformly — the paper does not bury this.

Seven reward policies: RewardMemory is the dividing line

Table 2 fixes the backbone and optimization budget and varies only the reward policy, across seven arms: VideoAlign only, a single provider (R1-Omni), random routing, round-robin routing, fixed fusion, Reflective Reward Gating, and full EmoReward with RewardMemory.

Adding RewardMemory brings the selectivity metric to 0.428 and both composite scores to core 0.922 and emo 0.884; the other six arms score 0.238/0.308, 0.302/0.340, 0.085/0.064, 0.266/0.272, 0.246/0.191, and 0.337/0.503, all below 0.922/0.884. The Reflect arm without memory reaches only 0.337 and 0.503 on those composites, though it takes a perfect 1.000 on the standalone EF preference dimension. The memory multipliers are also worth reading: macro expression and motion naturalness receive multipliers of 1.833 and 1.510, while three near-constant channels sit at 0.778.

Human evaluation: the bad-rating share says more than the good-rating share

Table 3 is 18 participants rating 30 prompts on three dimensions, using Bad/Normal/Good.

Emotion fidelity: EmoReward takes 60.0% "Good" against 53.0% for SAGE-GRPO; temporal coherence 54.3% against 51.5%; visual quality 57.2% against 45.0%. EmoReward also posts the lowest "Bad" share on all three: 15.4%, 20.2%, and 15.9%, versus 31.5%, 28.9%, and 30.9% for SAGE-GRPO.

The real finding is in the latter. Plain SAGE-GRPO scores more "Bad" than the un-finetuned base generator on all three dimensions (21.9%, 23.5%, 25.2%) — RL driven purely by general video rewards makes bad outputs more common, not less. EmoReward is what pushes that back down. The paper also notes these are descriptive results: participant- and prompt-level raw records are needed to quantify uncertainty.

Transfer to image generation, where the conclusion depends on the metric

The paper runs transfer experiments on SD 3.5 and FLUX. On SD 3.5 with DanceGRPO, HPSv3 improves with calibration added; on FLUX, though, the baseline already optimized for HPSv3 is not surpassed on that metric. The paper also notes that the SD 3.5 arms have unequal training budgets, which limits causal attribution, and that the first two FLUX baseline cases exist only as saved examples at 500 updates while the rest use 3000.

Limits the paper states itself

Appendix A is direct: the experiments evaluate a single reference-run-to-calibrated-run transition, not sustained improvement across repeated iterations; distribution shift, stale evidence, and feedback loops that reinforce a proxy's errors are all unresolved. Whether the same calibration rule transfers to language models or tool-use agents, the paper says, remains an empirical question.

Reproduction cost matters too: the official README recommends 80 GB of GPU memory per GPU across 32 GPUs (4 nodes × 8), Linux, PyTorch 2.6+. The repository's LICENSE is the Tencent Hunyuan community license rather than a standard OSI open-source license, and model weights are not distributed with Git. If a reward provider's weights are missing the pipeline degrades gracefully and redistributes that provider's weight, but CLIP-AUTT, R1-Omni, OpenFace-3.0 and the rest have to be fetched yourself to get full emotional fidelity.

So what

The value here is not "yet another video RL paper." It moves reward-system tuning out of manual engineering and into the training loop itself — the experience of the previous run becomes the basis for allocating trust in the next one.

But the boundaries matter: VideoAlign is both the target and the ruler, so an automatic metric rising does not mean emotion actually improved; the human study is 18 people and 30 prompts; and the benefit of repeated cross-run iteration is entirely unverified. For teams considering this, the honest order is to first get general-reward RL working and check whether bad outputs increased, then decide whether to add this calibration layer — because the paper's own data shows that what this layer actually changes is precisely which samples get judged bad.

Sources: paper https://arxiv.org/abs/2611.06518 , project page https://emoreward-project.github.io/ , official repository https://github.com/Tencent-Hunyuan/EmoReward .