基于混淆矩阵我们可以使用准确率（Precision）和召回率（Recall）来评价模型在不平衡数据上的分类精度。F-score（F1）和G-mean（GM）是准确率和召回率的调和平均值 [4,5]。MCC （Matthews correlation coefficient，即马修斯相关系数）考虑了各种样本的个数，是一个在类别平衡或不平衡下都可使用的评价指标。AUCPRC （Area Under Curve of Precision-Recall Curve）指准确率-召回率曲线下的面积。这些评价准则不会被不同类别中样本的数量所影响，因此通常被认为是“无偏的”，可以在类别不平衡的场景下使用。
需要注意的是一个常用的评价指标AUCROC（Area Under Receiver-Operator Characteristic curve）其实是有偏的，它 不适用 于不平衡场景下的模型评估。
Receiver Operator Characteristic (ROC) curves are commonly used to present results for binary decision problems in machine learning. However, when dealing with highly skewed datasets, Precision-Recall (PR) curves give a more informative picture of an algorithm’s performance.
— Davis J, et al. The Relationship Between Precision-Recall and ROC Curves, ICML 2006
 Gamberger D, Lavrac N, Groselj C. Experiments with noise filtering in a medical domain//ICML. 1999: 143-151.
 Graepel T, Candela J Q, Borchert T, et al. Web-scale bayesian click-through rate prediction for sponsored search advertising in microsoft’s bing search engine// Omnipress, 2010.
 Sun Y, Wong A K, Kamel M S. Classification of imbalanced data: A review. International Journal of Pattern Recognition and Artificial Intelligence, 2009, 23 (04):687-719.
 Powers D M. Evaluation: from precision, recall and f-measure to roc, informed-ness, markedness and correlation. 2011.
 Sokolova M, Japkowicz N, Szpakowicz S. Beyond accuracy, f-score and roc: a family of discriminant measures for performance evaluation//Australasian joint conference on artificial intelligence. Springer, 2006: 1015-1021.
 Boughorbel S, Jarray F, El-Anbari M. Optimal classifier for imbalanced data using matthews correlation coefficient metric. PloS one, 2017, 12(6):e0177678.
 Davis J, Goadrich M. The relationship between precision-recall and roc curves// Proceedings of the 23rd international conference on Machine learning. ACM, 2006: 233-240.
 García V, Sánchez J, Mollineda R. An empirical study of the behavior of classifiers on imbalanced and overlapped data sets//Iberoamerican Congress on Pattern Recognition. Springer, 2007: 397-406.
 Napierała K, Stefanowski J, Wilk S. Learning from imbalanced data in presence of noisy and borderline examples//International Conference on Rough Sets and Current Trends in Computing. Springer, 2010: 158-167.
 Prati R C, Batista G E, Monard M C. Learning with class skews and small dis- juncts//Brazilian Symposium on Artificial Intelligence. Springer, 2004: 296-306.
 Mani I, Zhang I. knn approach to unbalanced data distributions: a case study involving information extraction//Proceedings of workshop on learning from imbalanced datasets: volume 126. 2003.
 Wilson D L. Asymptotic properties of nearest neighbor rules using edited data. IEEE Transactions on Systems, Man, and Cybernetics, 1972(3):408-421.
Tomek I. Two modifications of cnn. IEEE Trans. Systems, Man and Cybernetics, 1976, 6:769-772.
 Chawla N V, Bowyer K W, Hall L O, et al. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 2002, 16:321-357.
 He H, Bai Y, Garcia E A, et al. Adasyn: Adaptive synthetic sampling approach for imbalanced learning// 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence). IEEE, 2008: 1322-1328.
 Han H, Wang W Y, Mao B H. Borderline-smote: a new over-sampling method in imbalanced data sets learning//International conference on intelligent computing. Springer, 2005: 878-887.
 Batista GE, Prati RC, Monard MC. A study of the behavior of several methods for balancing machine learning training data. ACM SIGKDD explorations newsletter, 2004, 6(1):20-29.
 Elkan C. The foundations of cost-sensitive learning//International joint conference on artificial intelligence: volume 17. Lawrence Erlbaum Associates Ltd, 2001:973-978.
 Liu X Y, Zhou Z H. The influence of class imbalance on cost-sensitive learning: An empirical study//Sixth International Conference on Data Mining (ICDM’06). IEEE, 2006: 970-974.
 Freund Y, Schapire R E. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences, 1997,55(1):119-139.
 Chawla N V, Lazarevic A, Hall L O, et al. Smoteboost: Improving prediction of the minority class in boosting//European conference on principles of data mining and knowledge discovery. Springer, 2003: 107-119.
 Wang S, Yao X. Diversity analysis on imbalanced data sets by using ensemble models//2009 IEEE Symposium on Computational Intelligence and Data Mining. IEEE, 2009: 324-331.
 Liu X Y, Wu J, Zhou Z H. Exploratory undersampling for class-imbalance learning. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2009, 39(2):539-550.
 He H, Garcia E A. Learning from imbalanced data. IEEE Transactions on Knowledge & Data Engineering, 2008(9):1263-1284.
 He H, Ma Y. Imbalanced learning: foundations, algorithms, and applications. John Wiley & Sons, 2013.
 Haixiang G,Yijing L,Shang J, et al. Learning from class-imbalanced data: Review of methods and applications. Expert Systems with Applications, 2017, 73:220-239
 Lemaître G, Nogueira F, Aridas C K. Imbalanced-learn: A python toolbox to tackle the curse of imbalanced datasets in machine learning. Journal of Machine Learning Research, 2017, 18(17):1-5. jmlr.org/papers/v18/16-.