QKeras model doesn't match the HLS4ML model

[alexberian]

I have made a 12bit QKeras model, and when I create my HLS4ML model from it I get different results.

# make the QKeras model
qkeras_model = simple_mnist_cnn()

# load data
(x_train,y_train),(x_test,y_test) = mnist_dataset()

# train it for just 1 epoch
qkeras_model.fit(x_train,y_train)

# create HLS model as per the tutorial
hls4ml.model.optimizer.OutputRoundingSaturationMode.layers = ['Activation']
hls4ml.model.optimizer.OutputRoundingSaturationMode.rounding_mode = 'AP_RND'
hls4ml.model.optimizer.OutputRoundingSaturationMode.saturation_mode = 'AP_SAT'

config = hls4ml.utils.config_from_keras_model(qkeras_model, granularity='name')
config['LayerName']['softmax']['exp_table_t'] = 'ap_fixed<12,1>'
config['LayerName']['softmax']['inv_table_t'] = 'ap_fixed<12,1>'
hls_model = hls4ml.converters.convert_from_keras_model(qkeras_model,
                                                       hls_config=config,
                                                       output_dir=os.path.join('.','hls4ml_conversion_dir'),
                                                       part='xcu250-figd2104-2L-e')
hls_model.compile()

# compare models
qkeras_acc = np.mean( np.argmax(y_test,axis=1) == \
             np.argmax(qkeras_model.predict(x_test),axis=1))
hls_acc =    np.mean( np.argmax(y_test,axis=1) == \
             np.argmax(hls_model.predict(x_test.astype(np.float32)),axis=1))
print('QKeras Accuracy: %.2f'%(qkeras_acc))
print('HLS4ML Accuracy: %.2f'%(hls_acc))

This code snippet yields the following;

QKeras Accuracy: .97
HLS4ML Accuracy: .22

I was under the impression that the point of designing/training a model with QKeras was to ensure that there is no loss in performance when converting into something hardware-compatible. Is there a mismatch between how QKeras and HLS4ML are doing the quantization? What should I do to make the models match?

[obsidian-1]

Did you find a work around for this?