FLOP Estimates Fail on Shared H200 Clusters

19 11 2026
FLOP Estimates Fail on Shared H200 Clusters

A preview of a paper analyzing 272 fine-tuning runs on NVIDIA H200 GPUs. It compares theoretical FLOP-based duration estimates against learned predictors. The FLOP baseline achieved an R² of 0.31 and a mean absolute error of 12.41 hours, while a CatBoost regressor reached R² 0.98 with a median error of 0.51 hours.

The learned model captures overheads like checkpointing and I/O contention that closed-form math misses. It is a preview, so the full methodology and code are not yet available. Useful for planning shared cluster capacity, but not for single-user local jobs.

more: https://huggingface.co/blog/Dharma-AI/predicting-fine-tuning-time

my other side: ivo.blogspot.com


Actions

Information

Leave a Reply