You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Couple of notes regarding schema registries:
1.
I'd like to propose that the authors of this spec re-consider how to detect breaking changes. Confluent's model of global backward/forward compatibility modes worked in the past, especially with things like Avro, but it's an essentially broken model nowadays, especially once you add Protobuf3 into the mix, which is much richer in it's specification.
I recommend having a look at this project: buf.build, specifically how they manage breaking changes in their CLI: https://docs.buf.build/breaking/rules
The main thing here is the configurability. They have multiple breaking change levels, but we can also provide a configuration file with more fine grained solutions. In a world of governance, multi language ecosystems, potentially mono vs multi repos, etc, it makes a lot more sense to have further control on what is the definition of a breaking schema.
Again, I recommend a divergence on how schemas are validated vs the implementation of Confluent's Schema Registry. Consider as well what a schema is, technically - a data model - and instead of checking if a schema is breaking against another, also validate if it is EQUIVALENT. This is an important distinction if we intend to support multi-languages and/or compilers.
Again, while this is much more prevalent in Protobuf, there are some issues with avro as well, where some compilers historically would remove comments, and the exact same schemas (one with comments, one without), would represent different versions of a schema. This means having to register multiple schemas in a schema registry in order to take with it.
With Protobuf, this becomes even worse, where depending on compilation rules you can get exactly the same data representation, but with slight differences (json_name options, comments vs no comments, re-ordering of fields, etc.) across language bindings. Any serious implementation of a protobuf3 schema registry should consider different but exactly equivalent proto3 schemas as having the same ID internally.
The text was updated successfully, but these errors were encountered:
Couple of notes regarding schema registries:
1.
I'd like to propose that the authors of this spec re-consider how to detect breaking changes. Confluent's model of global backward/forward compatibility modes worked in the past, especially with things like Avro, but it's an essentially broken model nowadays, especially once you add Protobuf3 into the mix, which is much richer in it's specification.
I recommend having a look at this project: buf.build, specifically how they manage breaking changes in their CLI: https://docs.buf.build/breaking/rules
The main thing here is the configurability. They have multiple breaking change levels, but we can also provide a configuration file with more fine grained solutions. In a world of governance, multi language ecosystems, potentially mono vs multi repos, etc, it makes a lot more sense to have further control on what is the definition of a breaking schema.
Again, I recommend a divergence on how schemas are validated vs the implementation of Confluent's Schema Registry. Consider as well what a schema is, technically - a data model - and instead of checking if a schema is breaking against another, also validate if it is EQUIVALENT. This is an important distinction if we intend to support multi-languages and/or compilers.
Again, while this is much more prevalent in Protobuf, there are some issues with avro as well, where some compilers historically would remove comments, and the exact same schemas (one with comments, one without), would represent different versions of a schema. This means having to register multiple schemas in a schema registry in order to take with it.
With Protobuf, this becomes even worse, where depending on compilation rules you can get exactly the same data representation, but with slight differences (json_name options, comments vs no comments, re-ordering of fields, etc.) across language bindings. Any serious implementation of a protobuf3 schema registry should consider different but exactly equivalent proto3 schemas as having the same ID internally.
The text was updated successfully, but these errors were encountered: