I noticed these two lines in the code eval/eval_quality.py
Alpha_Precision_all = qual_res['delta_precision_alpha_naive']
Beta_Recall_all = qual_res['delta_coverage_beta_naive']
But when I checked the paper that proposed alpha accuracy (Alaa, Ahmed, et al. "How faithful is your synthetic data? sample-level metrics for evaluating and auditing generative models." International Conference on Machine Learning. pmlr, 2022.) The paper appears to use the one-class classifier, this means that you should usequal_res['delta_precision_alpha_OC']
Why is delta_precision_alpha_naive used as the value for alpha precision here?
I noticed these two lines in the code
eval/eval_quality.pyBut when I checked the paper that proposed alpha accuracy (Alaa, Ahmed, et al. "How faithful is your synthetic data? sample-level metrics for evaluating and auditing generative models." International Conference on Machine Learning. pmlr, 2022.) The paper appears to use the one-class classifier, this means that you should use
qual_res['delta_precision_alpha_OC']Why is
delta_precision_alpha_naiveused as the value for alpha precision here?