MiniMax-H3 Acceleration LoRAs

What are these LoRAs?

alibaba-pai applies Parallel Decoding Distillation (PDD) (arXiv:2607.26004) to MiniMax-H3. Instead of a plain PEFT LoRA, PDD adds low-rank updates to the transformer backbone and repeats the two final heads (proj_out, audio_proj_out) once per interval of a length-N grid. Each generation step fuses one block of those heads into a single Euler velocity, so the effective number of transformer function evaluations is NFE = N / L — here N=32, L=4 → 8 steps.

Baseline · Turbo 4-step · Acc 8-step

Official test cases from Minimax-H3-Turbo, generated with a LoRA weight of 1.0 at both 4 and 8 NFE. Videos are streamed from the model repository — no GPU is needed to view them.

Reference input images

The four frames used to drive the Ref2VA example. Each defines a subject (<Subject 1..4>) in the generation prompt.

Run it with diffusers (≥ 0.40.0)

Set model_path and pdd_lora_path, then let apply_pdd_lora (from minimax_h3_pdd.py) inject the LoRA and derive the number of inference steps from its config.

FL2VA — text-to-video

import torch
from diffusers import ComponentsManager, ModularPipeline
from diffusers.utils.export_utils import encode_video

from minimax_h3_pdd import apply_pdd_lora  # from the model repo

model_path    = "MiniMaxAI/MiniMax-H3"
pdd_lora_path = "MiniMax-H3-FL2VA-Acc-8Step.safetensors"

prompt = ("[Shot 1] Cinematic wide shot, low angle, camera pushing in fast. "
          "A knight charges across a muddy tournament field...")

manager = ComponentsManager()
manager.enable_auto_cpu_offload(device="cuda", memory_reserve_margin="12GB")
pipeline = ModularPipeline.from_pretrained(model_path, workflow="t2va",
                                           components_manager=manager)
pipeline.load_components(dtype=torch.bfloat16,
                         pretrained_model_name_or_path=model_path)

# apply the Acceleration LoRA; returns the number of transformer FEs
nfe = apply_pdd_lora(pipeline.transformer, pdd_lora_path,
                     pipeline.scheduler.shift, pipeline.audio_scheduler.shift)

result = pipeline(
    prompt=prompt,
    height=704, width=1280, num_frames=124,
    num_inference_steps=nfe + 1,        # scheduler counts terminal sigma
    generator=torch.Generator().manual_seed(42),
    output_type="np",
    output=["videos", "audio", "sampling_rate"],
)

encode_video(result["videos"][0], fps=24, output_path="t2v.mp4",
             audio=result["audio"][0],
             audio_sample_rate=int(result["sampling_rate"]))

Ref2VA — multi-image reference → video

import torch
from diffusers import ComponentsManager, ModularPipeline
from diffusers.modular_pipelines.minimax_h3 import MiniMaxH3ImageReference
from diffusers.utils.export_utils import encode_video

from minimax_h3_pdd import apply_pdd_lora

model_path    = "MiniMaxAI/MiniMax-H3"
pdd_lora_path = "MiniMax-H3-Ref2VA-Acc-8Step.safetensors"

reference_images = [
    MiniMaxH3ImageReference.from_file(p)
    for p in ("asset/ref2va_test_5_1.jpg", "asset/ref2va_test_5_2.jpg",
              "asset/ref2va_test_5_3.jpg", "asset/ref2va_test_5_4.jpg")
]

manager = ComponentsManager()
manager.enable_auto_cpu_offload(device="cuda", memory_reserve_margin="12GB")
pipeline = ModularPipeline.from_pretrained(model_path, workflow="ref2va",
                                           components_manager=manager)
pipeline.load_components(dtype=torch.bfloat16,
                         pretrained_model_name_or_path=model_path)

nfe = apply_pdd_lora(pipeline.transformer_ref, pdd_lora_path,
                     pipeline.scheduler.shift, pipeline.audio_scheduler.shift)

result = pipeline(
    prompt=prompt,                  # your subject_definitions / summary text
    references=reference_images,
    height=768, width=1344, num_frames=243,
    num_inference_steps=nfe + 1,
    generator=torch.Generator().manual_seed(42),
    output_type="np",
    output=["videos", "audio", "sampling_rate"],
)