Which primary metric should you use?

You are creating a classification model for a banking company to identify possible instances of credit card fraud. You plan to create the model in Azure Machine Learning by using automated machine learning.

The training dataset that you are using is highly unbalanced.

You need to evaluate the classification model.

Which primary metric should you use?
A . normalized_mean_absolute_error
B . AUC_weighted
C . accuracy
D . normalized_root_mean_squared_error
E . spearman_correlation

View Answer

Answer: B

Explanation:

AUC_weighted is a Classification metric.

Note: AUC is the Area under the Receiver Operating Characteristic Curve. Weighted is the arithmetic mean of the score for each class, weighted by the number of true instances in each class.

Incorrect Answers:

A: normalized_mean_absolute_error is a regression metric, not a classification metric.

C: When comparing approaches to imbalanced classification problems, consider using metrics beyond accuracy such as recall, precision, and AUROC. It may be that switching the metric you optimize for during parameter selection or model selection is enough to provide desirable performance detecting the minority

class.

D: normalized_root_mean_squared_error is a regression metric, not a classification metric.

Reference: https://docs.microsoft.com/en-us/azure/machine-learning/how-to-understand-automated-ml