delay between clusters

[lut777]

I would like to ask the maintainer a few questions about the production environment, and would appreciate an answer.

1. Have you measured the time it takes for serviceimport endpoints in one cluster to respond to a change in the endpoint of a serviceexport in another cluster?
   This delay is important because it is the potential time when the service is unavailable.
2. When there is a massive endpoint reset in one cluster, how long does it take for the other cluster to finish the responding reset in their own cluster? How much bandwidth will it consume?

[tpantelis]

These questions are related to syncing information between clusters via the broker cluster which largely depends on the environment and network latency between clusters.

[nyechiel]

@lut777 we haven't measured these so far. Is there a concrete use-case or scale target you are worry about?

[lut777]

So is there a test result under average network conditions? No?
@nyechiel Yes, If there are 500 services in a cluster with rolling upgrades on the back end at the same time, how long will it take for another cluster to finish the upgrade?
This delay is critical cause it's svc's unavailability time. But we don't have the ability to change network conditions immediately. So the performance of lighthouse is important.
What if there are 5k services? It's very common these days to have thousands of services in a cluster.

[dfarrell07]

/cc @mkolesnik regarding your scale testing work.

[mkolesnik]

The scale testing work wasn't concluded and thus I have nothing to add at this time