Accurate Models of AMD Matrix Cores

Computer Science > Hardware Architecture Title:Accurate Models of AMD Matrix Cores Abstract:Matrix multipliers available on recent GPUs do not conform with the IEEE 754 floating point standard. Features of matrix multipliers differ across vendors and architectures of the same vendor, such as accumulator width, rounding behaviour, normalisation points, intermediate underflow and overflow logic, the handling of subnormals, and the treatment of special inputs. As a result, reproducibility of small matrix multiplier results across devices is not possible and cannot be achieved by software control. Implementation details of matrix multipliers are not documented, making it difficult to interpret discrepancies in the computed results. We characterise the numerical behaviour of matrix multipliers across three AMD GPU architectures: CDNA 1, CDNA 2, and CDNA 3, using the MI100, MI210/250, and MI300A/300X GPUs, respectively. We design test vectors to target numerical features for all supported input formats and provide the derivation and the reasoning for why each vector allows to determine a particular numerical feature based on the outputs of the devices. MATLAB-based software models of the matrix multipliers are then developed for each architecture and validated for bit-level reproducibility against hardware using a randomized test suite consisting of 10 million sets of random input vectors. To achieve this, we applied a previously developed technique to iteratively refine the accuracy of the models in a loop, by randomized testing followed by test-refinement until the model matches the hardware for every test case. Finally, as a proof of concept for what experimental research can be done with the models, we have utilised them in two demonstrative numerical applications, quantifying application-level accuracy differences between AMD matrix cores and the NVIDIA tensor cores. Submission history Access Paper: View PDF HTML (experimental) TeX Source Current browse context: References & Citations NASA ADS Google Scholar Semantic Scholar BibTeX formatted citation Bookmark Bibliographic and Citation Tools Code, Data and Media Associated with this Article Demos Recommenders and Search Tools Author Venue Institution Topic arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.