Support Vector Machines – Not for the faint-hearted
Support Vector Machines a.k.a SVM is a powerful algorithm used for regression and classification problems. Just like an Artificial Neural Network algorithm, SVMs also operate as a black box, that is, it’s difficult to explain how the input gets translated into the output. The term “Support Vectors” in SVM refers to the data points that lie on the boundaries that separate these data points (figure1).
Figure 1 – Overview
SVM operates by minimizing the error i.e. by maximizing the “margin” between the two classes within the dataset. The margin is simply the gap (distance) between the closest separating points and the hyperplane. A soft margin allows for some errors and is flexible, as opposed to a hard margin that is strict. A hyperplane is a plane in a higher dimension (a line in 2D or a plane in 3D, depending on the data, figure 2) that separates the two categories. This is the case of a linearly separable data, simply because it can be separated by a linear plane. Any point lying on the right of the line will be classified as black and any point lying on the left of the line will be predicted as blue.
Non-linearly separable data: In real life, the datasets are not always linearly separable. SVM converts the non-linear data into a higher dimension space and the kernel function tries to fit the data into a linear plane separation, called a feature space (figure 2 (b)). This process is the kernel trick. A great video explanation can be viewed here!
Figure 2 – Kernel Trick
SVM is widely used in image classification and in biological sciences, especially in protein categorization.
By: Mohna Sharma