Support Vector Machine

One of the most widely used supervised learning algorithms is the Support Vector Machine (SVM). It works based on dividing the given data set into two distinct categories with a well-defined gap. The dataset gets divided into many planes. Hyperplanes are used to identify the different categories. It is useful in both classification and regression. It is mostly used for classification.

Working of SVM

The main goal of the Support Vector Machine is to create the best fit line that can help in segregating the n-dimensional space into different classes. Segregation is important because it makes sure we assign the new data point in the correct class in the subsequent time. Furthermore, It chooses the extreme vectors or points which result in the creation of a hyperplane. The vectors or extreme points are known as support vectors. Let us understand this in more detail with the help of an example.

Example: Let’s say we have a goat that looks a lot like a baby cow and we want a model that can accurately distinguish it either as a goat or as a cow then we need to use the SVM algorithm to build the model. First, we have to train our model with a dataset of images of goats and cows because of this our model can learn different features of goats and cows. Then we have to test our model by providing the image of this strange goat. Since SVM creates a distinct boundary between the two classes i.e., goats, and cows and chooses the extreme support vectors. Finally, based on different support vectors it will classify this strange goat as a goat and will put this data point in the goat category.

Types of SVM

It is divided into two types:

Linear SVM

We use it for linearly separable data i.e., a dataset in which we can distinguish the dataset into two different classes with the help of only one straight hyperplane or a line. For this, we can consider our above example where we had a dataset that had only two different categories. Furthermore, We were able to differentiate it into two different classes i.e., goat and cow with a single hyperplane or a single straight line.

Non-Linear SVM

We use it for non-linearly separable data, i.e., a dataset which cannot distinguish it into different categories with a single hyperplane or a straight line.

Important Terminologies in SVM

Hyperplane

There can be multiple boundaries or lines to differentiate the classes in n-dimensional space. However, the best boundary or line for the model is called a hyperplane. For example, if we have 2 distinct features then we can separate them by a straight line but if we have 3 distinct features we will separate them by a 2-dimensional plane.

Margin

The closest points from different classes are known as support vectors and the distance between the hyperplane and these vectors is known as margin.

Optimal Hyperplane

The maximum margin of any hyperplane is known as an optimal hyperplane. We always prefer optimal hyperplanes over other hyperplanes.

Kernel Functions/Tricks

It classifies the non-linear data and it helps in transforming the non-linear data into linear data and then it draws a hyperplane.

Advantages & Disadvantages of SVM

Advantages

Regularization capabilities

Support Vector Machine has an L2 regularization feature which results in good generalization of capabilities which results in the prevention of overfitting models.

Efficiently handling non-linear data

It can efficiently handle non-linear data with the help of kernel tricks.

Resolves both classification & regression problems

It solves both problems as Support Vector Regression (SVR) solves regression problems while Support Vector Machine (SVM) solves classification problems.

Stability

Minute changes on the dataset do not affect the hyperplane of the model.

Disadvantages

Selecting an apt kernel function is difficult

While handling the non-linear data the selection of appropriate kernel function is tiresome. It is way complex and tricky. However, if you are using a high dimension kernel, it can result in generating too many support vectors which will result in the reduction of training speed in a drastic manner.

Memory Requirement is Extensive

SVM’s memory requirements and algorithmic complexity are very high. Since we have to store all the support vectors in the memory we will require a lot of memory. However, the number keeps growing abruptly with the training dataset size.

Needs Feature Scaling

You have to apply feature scaling on the variables before using SVM.

Training Time

It requires a lot of training time on large datasets.

Difficult in Interpretation

This model is quite difficult to interpret and understand, unlike decision trees.

Real-World Applications of SVM

Face Detection

Firstly, it classifies the parts of images as face and non-face and creates a square boundary around the face.

Hypertext and Text categorization

Moreover, it is for both transductive and inductive models. To classify documents into different classes we use training data. It categorizes while comparing the generated score with the threshold value.

Image Classification

In image classification, the Support Vector Machine provides better search accuracy. Moreover, it performs better according to accuracy in comparison to traditional query search-based searching techniques.

Bioinformatics

We can use SVM for the classifications of patients on different biological problems.

Handwriting Recognition

It can recognize handwritten characters that we use widely.

Summary

To sum up, this article focuses on the working of Support Vector Machine. Its advantages and disadvantages and real-world applications. SVM is an important supervise learning algorithm in the world of Machine Learning. It is based on statistical learning frameworks and has one of the most accurate prediction methods.

SOURCES

For more articles, CLICK HERE.