How are submissions judged?

[Samhori]

I was wondering what criteria submissions are judged by? If it is a metric similar to accuracy, which is what seems to be what is used at the moment, then the fraction of the test data with an anomaly matters a lot which will mean that scoring well will rely on accurately guessing how many anomalies were placed in the test set. The problem description suggest we should provide "an array of predicted probabilities in the range [0, 1]" so I believe that a method like cross-entropy or AUROC is being applied instead, but it would be very helpful to understand exactly what is being provided.

Also, the description of the Sine-Gaussian data says that "these are generic low-frequency signals used to represent potential gravitational wave sources that do not fit into the well-understood categories like BBH" which seems to imply that the aim of this challenge is to identify sine-gaussians. My understanding is that this is wrong and that in the test set there will be unspecified anomalies so that submissions are judged on their ability to identify a completely generic anomaly, but could you clarify whether this understanding is correct?