“Property trampolines”, or get_type_override() and PYBIND11_OVERRIDE_... macros for properties

[ct-ms]

Hi!

In our codebase, there are some abstract base classes which are currently implemented in Python code, but which we would like to have the option of subclassing and using in both Python or C++ going forward. We are therefore converting the ABCs themselves to C++, with Python bindings via pybind11, alongside trampoline classes with appropriate PYBIND11_OVERRIDE method bodies.

This works very well for the most part, but we are struggling with those methods which are marked @property in the current Python interface code. When we use PYBIND11_OVERRIDE for these, we send the program into an infinite loop. My understanding is that whenever the method from the trampoline class runs, the getattr() call for the member name in get_type_override() does not result in a bound method object as it does for non-@property methods, but instead calls the C++ method body of the pybind11-bound property, which of course ends up back in the trampoline method, and the cycle is complete.

It would seem to me like the solution to this would be something like a PYBIND11_OVERRIDE_PROPERTY set of macros that use a slightly different version of get_type_override() under the hood. This would not effectively getattr(self, member_name), as that already evaluates the property thanks to the magic of property.__get__(), but instead do something like getattr(getattr(self.__class__, member_name), "fget") (or "fset" for writing to a property) to locate the Python-based override, if any.

Right now, replacing our read-only properties with getter methods seems like the only way to get around this if we don’t want to maintain our own trampoline mechanism. But this is of course an API change (and a leak in the abstraction provided by pybind11), while we would prefer a transparent transition from a Python-only to a C++-and-Python interface, allowing our downstream users to both use and override the attributes as they’ve been able to until now. I believe that this could be of general interest to pybind11 users other than us.

Does the idea make sense, or is there perhaps already a way to do this that we’re missing? Any input is appreciated.

Thank you!

[Skylion007]

Currently, PYBIND11_OVERRIDE macro is super hacky and tries to autodetect loops like this, but occasionally fails like so. The strategy you suggested seems much cleaner and we would welcome any PRs that improves on our trampoline logic prevent these infinite loops.