General question about output of p-values using rcorr and multicomp functions

[emward1]

Hi! I am trying to run a correlation matrix with sample size n = 433 for 15 features (3 dependent variables (test types) and 12 independent variables (survey results, age, etc.). While trying to adjust p-values for multiple comparisons (fdr_bh - Benjamini–Hochberg), it is not clear to me what exactly the outputs of rcorr and multicomp are...

If I'm understanding correctly, I believe the output of multicomp(..., method='fdr_bh') gives the adjusted p-values, which I would then compare to alpha = 0.05 to determine significance of each correlation. But how exactly would this handle a np.array (15 x 15) of p-values?

And for rcorr, I understand that you can input a dataframe (matrix-like) and display the p-values on the upper triangle of the correlation matrix. However, the language in the description and example confuses me. It says it will "Display the raw p-values." Does that mean these p-values are NOT the adjusted (FDR-Benjamini in my case) p-values that I am trying to find? Should these p-values should be compared to alpha = 0.05 to determine significance?

Lastly, when I compare the p-values from multicomp and rcorr (both method='fdr_bh'), the resulting p-values are quite different. My guess is that has to do with multicomp taking in 15^2 "features" while rcorr calculates 15 choose 2 "features," but again, I am not quite sure. Any responses and clarification is greatly appreciated!

[raphaelvallat]

Hi @emward1,

1. multicomp always flatten the input array before computing the corrected p-values (see here).

I believe the output of multicomp(..., method='fdr_bh') gives the adjusted p-values, which I would then compare to alpha = 0.05 to determine significance of each correlation

That's correct

It says it will "Display the raw p-values."

I agree that the language is confusing. "Raw" in this context means that the numerical values will be shown as opposed to stars (e.g. *, ***). If padjust is not None, then it will show the adjusted p-values.

when I compare the p-values from multicomp and rcorr (both method='fdr_bh'), the resulting p-values are quite different.

I think that your interpretation is correct, but feel free to share a more detailed example if my answer doesn't help.

Thanks,
Raphael

[emward1]

Thanks Raphael! I think that clears everything up for me! I appreciate your response and the work you put into the package! :)

Best,
Erica