How do you compute the cosine similarity for evaluating the FP8 quantization quality?
Many thanks for the great work!
I was wondering how do you compute the cosine similarity for evaluating the FP8 quantization quality during training. is it per tensor flatten then dot product or per row DP or per column DP?
Thank you for your attention!
Taking Y = X @ W.T as an example, we use the mean cosine similarity of each row in the tensor (X or W) before and after FP8 quantization as the distortion metric. During this process, rows with all zero elements will be ignored.
Thank you for your attention!
Taking Y = X @ W.T as an example, we use the mean cosine similarity of each row in the tensor (X or W) before and after FP8 quantization as the distortion metric. During this process, rows with all zero elements will be ignored.
Thank you for the reply!
So I can understand this as cosine similarity of the rows of X and columns of W.T, that perform the Matrix multiply accumulation process?
May I ask if you have any underlying reason that cosine similarity serves as a better metric of quantization quality than Signal-to-Noise Ratio (SNR)?