Questions about T-CREx generator

[pat-alt]

Hi @tombewley, hope you're well!

We met at ICML in Vienna this summer and last week I've finally come around to adding T-CREx here to our package (#476). I've focused on merely reproducing the worked example in the paper, so there are some missing pieces still (#473) but the basic implementation is done and documented here.

I thought I'd open this thread in the hope that you might be able to lend a second pair of eyes, ensure that everything is done correctly and discuss some questions that have come up along the way.

Questions

• Q1: Would you be able to share the exact specifications for the worked example in your paper or perhaps even the dataset directly? I've tried to reproduce it as closely as possible, but I get a different surrogate tree and hence different CREs.
• Q2: I'm not sure I understand Equation (13) in the paper correctly - should this not be the intersection of bounds? (since we want unique bounds only)
• Q3: Relatedly, just following Equation (13), should it not be $(2|R^{\max}| - 1)^D$ for the worst-case grid size (instead of $+1$)? In any case though, I don't understand how you get to 27 for the example in Figure 2.

Regarding Q2 and Q3 I might just be missing something. I might have more questions, when I find an opportunity to dive deeper and come back to #473. If you don't mind, I'll ping you here again. Finally, if you would like to contribute here, you are of course more than welcome, although I understand you probably have lots of other things going on.

In any case, very interesting work - thanks for sharing it at ICML.

[tombewley]

Hi @pat-alt, it's great to hear from you, and I really appreciate the effort you're making in this implementation! I'm very happy to answer these questions (and any more you have going forward):

• Q1: For this simple example, we used make_moons from the scikit-learn Python package. Specifically, we used make_moons(n_samples=5000, noise=0.3, random_state=0). I'm not sure whether you can call that function identically in Julia, especially as the seeding may behave differently, but you probably know better than me.
• Q2: Equation (13) basically says: for each feature dimension $d$, look at all the lower and upper bounds of the maximal-valid rules along that feature and aggregate them into one big set by taking the union. Using the notation of a set (curly brackets) means we're implicitly taking the unique values because all elements of a set are unique. Additionally, we always include $\pm\infty$ in this set of bounds (they might be in the set anyway, if one or more of the maximal-valid rules extends to infinity along feature $d$).
• Q3: The reason why it's $(2|R^\max|+1)^D$, rather than $-1$, is exactly because of this inclusion of $\pm\infty$. The worst case occurs when (a) there is no duplication of bounds across the maximal-valid rules and (b) none of these rules extend to infinity. In this case, for each feature $d$, each of the $|R^\max|$ rules contributes $2$ bounds to the set, then we add $2$ more for $\pm\infty$. This gives $(2|R^\max|+2)^D$ bounds across all features, and hence $(2|R^\max|+1)^D$ cells because there's always one fewer cell than the number of bounds in each direction (e.g. to draw a 3x3 grid, you need to draw 4 horizontal lines and 4 vertical lines).

Hopefully this clears things up a bit! Let me know if you need more clarification.

A small request from me too: could you add the citation for the accepted ICML paper in the References section of your documentation page? The ICML URL is here.

[pat-alt]

Thanks for the quick and detailed response, this makes things much clearer. I've updated the docs to include the reference to the ICML paper. I'll also use scikit-learn directly to generate the data and then the remaining differences (if any) should only be due to differences DecisionTree implementation.

Thanks again for your quick response, it's much appreciated! I'll keep this discussion open for the future 😃