simulator: add counterexample minimization

[alpaylan]

This PR introduces counterexample minimization(shrinking) in the simulator. It will require changes to the current structure in various places, so I've opened it as a draft PR for now, in order to not overwhelm the reviewers all at once.

• Turn interactions plans into sequences of properties instead of sequences of interactions, adding a semantic layer.
• Add assumptions to the properties, rendering a property invalid if its assumptions are not valid.
• Record the failure point in a failing assertion, as shrinking by definition works when the same assertion fails for a smaller input.
• Add shrinking at three levels, top level(removing whole properties), property level(removing interactions within properties), interaction level(shrinking values in interactions to smaller ones).
• Add marauders as a dev dependency, inject custom mutations to a testing branch for evaluating the simulator performance.
• Integrate the simulator evaluation with the CI.

[pereman2]

This looks quite cool @alpaylan. Could you post some literature you think is relevant for this?

[alpaylan]

This looks quite cool @alpaylan. Could you post some literature you think is relevant for this?

Thanks @pereman2 ! Of course, here are some rather informal writing discussing different shrinking strategies(warning: they're quite opinionated with respect to author's position, which I don't agree)

• https://hypothesis.works/articles/integrated-shrinking/
• https://hypothesis.works/articles/compositional-shrinking/

Two other informal articles,

• https://propertesting.com/book_shrinking.html
• https://well-typed.com/blog/2023/04/falsify/

ECOOP20 paper from the hypothesis author,

• https://drmaciver.github.io/papers/reduction-via-generation-preview.pdf

I view shrinking very related to delta debugging, which I think is a lot more used in the literature. I have a submission that discusses shrinking too, I'm not sure if sharing it here breaks double blind, but I can send it over email if you would like.

[jussisaurio]

I thought this page had a nice layman definition for shrinking:

In property-based testing, the initially found failing case may contain a lot of complexity that does actually cause the test to fail. Shrinking is the mechanism through which a property-based testing framework can simplify failing cases in order to find out the minimal reproducible case is.

We could even add it to our code

[jussisaurio]

Btw you can just do sth like

_ => unreachable!("unexpected: ...")

to reduce some lines-of-code noise from the pattern matching. or alternatively

if let Self::Query(query) = self {
  ...
} else {
  unreachable!(...)
}

[alpaylan]

That's a nice touch, adding it in the next commit.

[alpaylan]

The shrinking is a bit harder than I hoped for, particularly due to the fact that we cannot shrink closures, which limits our ability to minimize aggressively by removing columns from tables, because we wouldn't be able to modify the assertion that checks it.

The solution is to turn the assertions themselves into a small DSL, but at that point it goes into too much research, which I don't think is the right choice for now. I've instead focused on other shrinking mechanisms;

• Removing properties after the failing one,
• Removing properties that do not refer to the tables in the failing interaction,
• Removing extensional parts in the property(I'll also add these with some bit of explanation, probably today)