ROCm 7.2.4 has been released to enhance performance for AI inference workloads on AMD Instinct GPUs, particularly focusing on reducing latency and improving stability. Key updates include optimizations to hipGraphLaunch dispatch delays, fixes for memory copy regressions in CPX mode on MI300 series cards, and improved profiling accuracy that eliminates phantom idle gaps during vLLM execution. The new version also optimizes MIGraphX to bypass redundant tensor copies for small batch sizes, although users should be aware of a minor throughput dip for int8-quantized models. Administrators are advised to verify firmware compatibility and transition away from deprecated tools before the 2026 end-of-support deadline to ensure continued operational efficiency

Fix AI Inference Latency and Stability on AMD Instinct GPUs with ROCm 7.2.4

ROCm 7.2.4 delivers targeted performance patches for AI inference workloads running on AMD Instinct GPUs, focusing on latency reduction and stability improvements. The update trims hipGraphLaunch dispatch delays, fixes a memory copy regression in CPX mode for MI300 series cards, and cleans up profiling traces that previously showed phantom idle gaps during vLLM execution. MIGraphX gains optimizations to skip redundant tensor copies at small batch sizes, though int8-quantized models may experience a minor throughput dip until AMD addresses it. Server administrators should verify firmware compatibility before upgrading and plan their migration away from deprecated profiling tools well ahead of the 2026 end-of-support deadline.

Fix AI Inference Latency and Stability on AMD Instinct GPUs with ROCm 7.2.4 @ Linux Compatible