There is xquant which is the quantization tool available from spacemit, however it builds on ppq, is it also possible to simply just use ppq, or will there be some unforeseen problems, or performance issues?
For reference here is the github of the ppq tool:
PPQ has done a very good job in quantization framework, and we have borrowed its software design for building SpacemiT’s quantization tool xquant, but as a product-level tool chain, PPQ has a lot of flaws and shortcomings in ONNX model quantization, our engineers have done a better design of quantization pipeline, including asymmetric KL correction, self-adaptive LSQ, presets, and more user-friendly customized quantization strategy intervention,more generalized ONNX Function OP expression for exporting ONNX files. In summary, our xquant PPQ-based quantization framework is so well-designed that you will make fewer errors, get better quantization accuracy, and have a more end-to-end experience!
Additionally, if you have very strong quantization tool knowledge, you can use PPQ for custom quantization of ONNX models, but its export format may not be recognized by ONNXRuntime as a quantization operator for acceleration.We provide xquant as a python wheel package, you can get the full source code of xquant in your environment, and we may subsequently consider pushing it to the open source community. If you encounter problems and have further suggestions, please let us know!