Recently MLPerf v1.0, an industry standard on measuring AI performance in standard models, was released. MLPerf Training measures the time it takes to train machine learning models to a standard quality target in a variety of tasks including image classification, object detection, NLP, recommendation, and reinforcement learning. Processors from the likes of Nvidia, Google, Graphcore, AMD, Intel, Habana, and Huawei all made appearances from a variety of server OEMs.
"One of the negatives of MLPerf is that many firms will only submit results in the few models where they excel, and they will simply leave the rest empty."
Wish they said "we'll only publish your results if you submit for all categories".
This isn't directly mentioned in the article (and I don't think they're in MLPerf either), but do you have any analysis on SambaNova's hardware?
We do, not anything public though.
"One of the negatives of MLPerf is that many firms will only submit results in the few models where they excel, and they will simply leave the rest empty."
Wish they said "we'll only publish your results if you submit for all categories".
Noone would submit at all besides Nvidia in that case.
Fair is fair ¯\_(ツ)_/¯
Has there been any comparison done with Brainchip's Akida Neural Processor (https://brainchipinc.com/products/)?
Clients and employees of SemiAnalysis may hold positions in companies referenced in this article. Isn't that obvious?
We hold no position in Nvidia. Some of our clients may. That line is included in every article.