Multi-Class classification are those predictive modeling problems where examples are assigned one of more than two classes.
The problem is often framed as predicting an integer value, where each class is assigned a unique integer value from 0 to (num_classes – 1). The problem is often implemented as predicting the probability of the example belonging to each known class.
In this section, we will investigate loss functions that are appropriate for multi-class classification predictive modeling problems.
We will use the blobs problem as the basis for the investigation. The make_blobs() function provided by the scikit-learn provides a way to generate examples given a specified number of classes and input features. We will use this function to generate 1,000 examples for a 3-class classification problem with 2 input variables. The pseudorandom number generator will be seeded consistently so that the same 1,000 examples are generated each time the code is run.
1 2 | # generate dataset X, y = make_blobs(n_samples=1000, centers=3, n_features=2, cluster_std=2, random_state=2) |
The two input variables can be taken as x and y coordinates for points on a two-dimensional plane.
The example below creates a scatter plot of the entire dataset coloring points by their class membership.
1 2 3 4 5 6 7 8 9 10 11 | # scatter plot of blobs dataset from sklearn.datasets.samples_generator import make_blobs from numpy import where from matplotlib import pyplot # generate dataset X, y = make_blobs(n_samples=1000, centers=3, n_features=2, cluster_std=2, random_state=2) # select indices of points with each class label for i in range(3): samples_ix = where(y == i) pyplot.scatter(X[samples_ix, 0], X[samples_ix, 1]) pyplot.show() |
Running the example creates a scatter plot showing the 1,000 examples in the dataset with examples belonging to the 0, 1, and 2 classes colors blue, orange, and green respectively.
Scatter Plot of Examples Generated from the Blobs Multi-Class Classification Problem
The input features are Gaussian and could benefit from standardization; nevertheless, we will keep the values unscaled in this example for brevity.
The dataset will be split evenly between train and test sets.
1 2 3 4 | # split into train and test n_train = 500 trainX, testX = X[:n_train, :], X[n_train:, :] trainy, testy = y[:n_train], y[n_train:] |
A small MLP model will be used as the basis for exploring loss functions.
The model expects two input variables, has 50 nodes in the hidden layer and the rectified linear activation function, and an output layer that must be customized based on the selection of the loss function.
1 2 3 4 | # define model model = Sequential() model.add(Dense(50, input_dim=2, activation='relu', kernel_initializer='he_uniform')) model.add(Dense(..., activation='...')) |
The model is fit using stochastic gradient descent with a sensible default learning rate of 0.01 and a momentum of 0.9.
1 2 3 | # compile model opt = SGD(lr=0.01, momentum=0.9) model.compile(loss='...', optimizer=opt, metrics=['accuracy']) |
The model will be fit for 100 epochs on the training dataset and the test dataset will be used as a validation dataset, allowing us to evaluate both loss and classification accuracy on the train and test sets at the end of each training epoch and draw learning curves.
1 2 | # fit model history = model.fit(trainX, trainy, validation_data=(testX, testy), epochs=100, verbose=0) |
Now that we have the basis of a problem and model, we can take a look evaluating three common loss functions that are appropriate for a multi-class classification predictive modeling problem.
Although an MLP is used in these examples, the same loss functions can be used when training CNN and RNN models for multi-class classification