Originally published at: NVIDIA, Arm, and Intel Publish FP8 Specification for Standardization as an Interchange Format for AI | NVIDIA Technical Blog
NVIDIA, Arm, and Intel jointly propose a FP8 format for standardization as an interchange format for AI and improve computational efficiency.
NVIDIA, Arm, and Intel Publish FP8 Specification for Standardization as an Interchange Format for AI
An unintended (or intended?) method of feature selection at each layer? Features (at each layer) < magnitude for representation are effectively zero, removing them from the “feature set” input to the next layer. Lowest magnitude features are typically noisy (analogy to recursive feature elimination (RFE) for linear SVM’s). So maybe the results indicate a bit of tradeoff between accuracy of hyperplane orientation at each node, and the practical effectiveness of feature selection. (I used to hate the idea of feature selection, but now use it heavily, including in my latest classifier, currently in use in a manufacturing quality control application for a high end sensor.)