What is difference of torch.quantization and onnx.quantization for speed and accuracy ?

[demuxin]

For the effect of quantization (speed and accuracy), which quantization method is recommended?

[cjluo-omniml]

Ideally they should yield to the same accuracy. Speed-wise:

The torch.quantization: torch model->quantized torch model->quantized onnx->TRT
The onnx.quantization: torch model->onnx model-> quantized onnx->TRT

It really depends on how compatible the quantized torch->onnx conversion with the TRT you use. In most cases, the onnx.quantization may have better perf because we baby sit the onnx dqd node insertion while we cannot do it if we quantize in torch.