Model Evaluation - Tour of Model Evaluation Metrics

A classifier is only as good as the metric used to evaluate it. If you choose the wrong metric to evaluate your models, you are likely to choose a poor model.

In this tutorial, you will discover metrics that you can use for imbalanced classification. After completing this tutorial, you will know:

1. About the challenge of choosing metrics for classification, and how it is particularly difficult when there is a skewed class distribution.
2. How there are three main types of metrics for evaluating classifier models, referred to as rank, threshold, and probability.
3. How to choose a metric for imbalanced classification if you don’t know where to start.

Foundation - Challenge of Imbalanced Classification

In this tutorial, you will discover data characteristics that compound the challenge of imbalanced classification. After completing this tutorial, you will know:

1. Imbalanced classification is specifically hard because of the severely skewed class distribution and the unequal misclassification costs.
2. The difficulty of imbalanced classification is compounded by properties such as dataset size, label noise, and data distribution.
3. How to develop an intuition for the compounding effects on modeling difficulty posed by different dataset properties.

Foundation - Intuition for Imbalanced Classification

In this tutorial, you will discover how to develop a practical intuition for imbalanced and highly skewed class distributions.

After completing this tutorial, you will know:

1. How to create a synthetic dataset for binary classification and plot the examples by class.
2. How to create synthetic classification datasets with any given class distribution.
3. How different skewed class distributions actually look in practice.

Foundation - What is Imbalanced Classification?

Imbalanced classification is the problem of classification when there is an unequal distribution of classes in the training dataset.

The imbalance in the class distribution may vary, but a severe imbalance is more challenging to model and may require specialized techniques.

Many real-world classification problems have an imbalanced class distribution, such as fraud detection, spam detection, and churn prediction.

Project - Regression Machine Learning Case Study

How do you work through a predictive modeling machine learning problem end-to-end? In this lesson you will work through a case study regression predictive modeling problem in Python including each step of the applied machine learning process. After completing this project, you will know:

• How to work through a regression predictive modeling problem end-to-end
• How to use data transforms to improve model performance 
• How to use algorithm tuning to improve model performance
• How to use ensemble methods and tuning of ensemble methods to improve model performance