Delivery order with parallel nested defers

[yaacovCR]

In #80, we decided that nested defers, even at the same level, should always be delivered "in order," from the outermost defer to the innermost. What if there are multiple sibling nested defers? Should all of the outermost defers be delivered before any of the innermost?

For example:

query GenericExample {
  me {
    fastField
    ... @defer(label: "OuterA") { intermediateFieldA ... @defer(label: "InnerA") { slowFieldA } }
    ... @defer(label: "OuterB") { intermediateFieldB ... @defer(label: "InnerB") { slowFieldB } }
  }
}

Once intermediateFieldA is completed, OuterA can be delivered. At that point, can slowFieldA begin executing (assuming no early execution) and be delivered, if it chances to complete before intermediateB?

Crucially, as mentioned in the early execution discussion, slowFieldA may in fact interfere with or delay execution of intermediateFieldB, so one might argue for the delivery of all singly-deferred fragments as an initial "stage," followed by all doubly deferred fragments in the next "stage," etc.

On the other hand, consider the following example:

query OuterComponentsOfAsymmetricImportance{
  me {
    userName
    ... @defer(label: "SemiCriticalUserDetails") { mostCriticalField ... @defer(label: "SlightlyLessCritical") { slightlyLessCriticalField} }
    ... @defer(label: "UnimportantUserDetails") { unimportantField ... @defer(label: "ExpensiveFields") { expensiveUnimportantField} }
  }
}

In a fragment-oriented component-based approach, the developer of the SemiCriticalUserDetails component may attempt to use @defer to speed up initial rendering of their component, without knowing that their component was being deferred at all, let alone that the use of @defer within their component may cause the deferred fields in their critical component to have to wait for all other singly deferred components to complete.

The problem is that although the degree of nesting tells us the priority of a fragment with respect to the fragment's branch, we cannot assume at all that the degree of nesting corresponds to the fragment's priority across branches.

Looking forward, a by argument for the @defer argument can potentially be used (at a later stage, as a feature enhancement) to add some specificity and allow comparison across branches. (Perhaps a priority argument could alternatively be used to assign an absolute priority, but the idea is similar.) The above example could be rewritten something like this:

query OuterComponentsOfAsymmetricImportance{
  me {
    userName
    ... @defer(by: 1, label: "SemiCriticalUserDetails") { mostCriticalField ... @defer(by: 5, label: "SlightlyLessCritical") { slightlyLessCriticalField} }
    ... @defer(by: 50, label: "UnimportantUserDetails") { unimportantField ... @defer(by: 100, label: "ExpensiveFields") { expensiveUnimportantField} }
  }
}

This would require best practices as to deferBy intervals within an organization, but seems like a feasible path forward.

Presumably, without the use of this argument, ie in the initial example, a constant default of some number would be used, and so we would get the staged approach.

Another question is how this interacts with a stream being run in parallel to a few nested defers? How many streaming elements should we allow to run before running a nested defer? Should we make no assumptions in this regard at all?

Open for discussion!

[benjie]

Keep it simple; just do what the user would expect: siblings are independent, children are dependent. We can add a priority system later if it turns out to be a need (I strongly doubt it will).

query GenericExample {
  me {
    fastField
    ... @defer(label: "OuterA") { intermediateFieldA ... @defer(label: "InnerA") { slowFieldA } }
    ... @defer(label: "OuterB") { intermediateFieldB ... @defer(label: "InnerB") { slowFieldB } }
  }
}

• OuterA and OuterB can run in parallel
• Once intermediateFieldA has been sent, slowFieldA can execute/be sent
• Independently, once intermediateFieldB has been sent, slowFieldB can execute/be sent

I am still of the belief that one level of @defer / @stream is where the biggest value comes from incremental delivery (giving easy data up front, and deferring the more expensive) - after that I think there's diminishing returns. We should protect that first layer so that the initial data is returned as early as possible, but after that if things run non-optimally it's not too big a deal IMO. I, personally, would probably choose to inline everything after the first level of defers in general 🤷‍♂️

[benjie]

I see what you're saying, but I think you're still overthinking it and/or expecting users to overthink it. The simple rule that sibling defers are independent and nested defers are dependent is quite easy to follow - it has a strong parallel with how selection sets work in GraphQL already (sibling fields run in parallel, a field must complete before its child selection set is evaluated).

What a user wants to happen, and what they would expect to happen, are two different things. The entire OuterA and InnerA tree might complete before OuterB even sends the first response - and that's fine. Even if InnerA and InnerB have the same field in them, that's no problem - it just means that when OuterB completes it doesn't need to add any more pendings.

In my version there was no need to use callbacks, though we did temporarily store the parent objects for selection sets that have not been fully evaluated yet.

[yaacovCR]

I agree that the strong parallel to how selection sets work is alluring.

On callbacks, in your version -- as far as I can tell -- there are no callbacks, but you also have another round of unnecessary execution deferral when an in a situation of some shared and some unshared execution groups, i.e. an operation like this:

{
  ... @defer { someField { slowField a } }
  ... @defer { someField { slowField b } }
}

As far as I can tell, in the the field info for a and b is returned alongside the result of the execution group with slowField, with the new execution groups for a and b only created and initiated when the group for slowField completes. I'm working off of the incremental4 branch. Is that about right? Do you consider it a non-problem?

-- I'm using callbacks to avoid that.

As an aside, if there is always early execution, the spec problem with the example I gave is mitigated, giving it again here:

{
 ... @defer(label: "OuterA") { someFieldA ... @defer(label: "InnerA") { x { a b } } }
 ... @defer(label: "OuterB") { someFieldB ... @defer(label: "InnerB") { x { a c } } }
}

If there is always early execution or, importantly, even just early execution once the initial result has returned, then we can treat b and c the same, initiating execution of both of them immediately, as if InnerA is executing, it means the initial result has been sent, and early execution is allowed, so that InnerB has also started executing. [Our current implementation which always performs execution therefore does not contain a bug, although we probably want to rewrite it for other reasons, namely as to whether early execution should be enabled by default.]

[benjie]

As far as I can tell, in the the field info for a and b is returned alongside the result of the execution group with slowField, with the new execution groups for a and b only created and initiated when the group for slowField completes. I'm working off of the incremental4 branch. Is that about right? Do you consider it a non-problem?

Yes, that's one of the trade-offs of my solution. I don't expect it to be particularly problematic in real-world use cases, but I could be wrong 🤷‍♂️

[yaacovCR]

Our current implementation which always performs execution therefore does not contain a bug, although we probably want to rewrite it for other reasons, namely as to whether early execution should be enabled by default.]

Thinking about this more, early execution does not save us from this bug. If someFieldA has completed, then we should start executing { x { a b } but { x { c } should have to wait a tick -- because our version of early execution involves the delay of a tick.

[yaacovCR]

Thinking about this, yet even more, I think I was right the first time. Because we always do early execution, we can’t speak of starting execution, everything has already been set up to execute either immediately or one tick later, so nothing needs to change. At least I think.