Dimensionality Reduction in ML

“Machine Learning is the future of humans and the universe.”

At the time of final classification, there are too many factors present. These factors are variables. If we increase the number of features it becomes difficult to visualize the training data. Moreover, it makes working on the training set harder. Sometimes the features are correlated and hence they are redundant. To solve this Dimensionality Reduction comes into the picture. It is a technique that results in a lesser number of input variables in the dataset. High-dimensionality measurements and dimensionality reduction procedures are frequently utilized for information representation. In any case, these strategies can be utilized in applied ML to improve the classification or regression accuracy of the dataset. As a result, it makes a better fit for the predictive model.

Components of Dimensionality Reduction

There are two components of Dimensionality reduction:

Feature Selection

It is a process of selecting a subset of features that seems more relevant while using the model. It simplifies the models which will be easy to interpret by users. Moreover, it avoids the curse of dimensionality. Mostly, it involves three ways i.e., Embedded, Filter & Wrapper.

Feature Extraction

It reduces the set of raw data into manageable groups for processing. One of the most important characteristics of these datasets is that they require a lot of computing resources to process. It reduces the amount of data but it still completely and accurately describes the original dataset. 

Why is Dimensionality reduction important?

The issue of undesirable expansions in measurement is firmly identified with others. That was to the obsession with estimating/recording information at a far granular level than it was done in the past. This is not the slightest bit recommending that this is a new issue. It has begun acquiring significance recently because of a flood in information. The sensors used in businesses are widely expanding. These sensors constantly record information and store it for investigation at a later point. As a result of so much data, there can be redundancy.

Methods to perform Dimensionality Reduction

• Missing Value

While investigating the dataset, on the off chance that we experience missing qualities, what do we do? Our initial step ought to be to distinguish the explanation. Then, at that point, need to attribute missing qualities/drop factors utilizing suitable strategies. Be that as it may, imagine a scenario in which we have too many missing qualities. Would it be advisable for us to attribute missing qualities or drop the factors? At that time we should use mean, mode, and median values and we replace them with missing values.

• Low Variance

How about we think about a situation where we have a steady factor (all perceptions have a similar worth, 5) in our dataset. Do you figure it can work on the force of the model? NOT, because it has zero difference.

• Decision Trees 

We can utilize it as an extreme answer for tackling different difficulties. Like missing values, outliers, and recognizing significant variables. Many data scientists utilized decision trees and it functioned admirably for them.  Indecision trees, internal nodes represent the features of the dataset. As well as branches represent the decision rules while each leaf node represents the outcome. 

• Random Forest 

Random Forest is an ensemble of decision trees. Moreover, it develops a group of decision trees to classify data objects.  One of the main parameters of random forest is n_estimator because it decides the number of decision trees. It can perform both classification and regression. The higher the number of trees in the forest, the higher is accuracy. The higher number of trees in the random forest also prevents the problem of overfitting. Random forest is more biased for the variables that have more no. of distinct values. As a result, it favors numerical variables more than binary or categorical values. 

• High Correlation 

Dimensions showing higher correlation can drop down the presentation of a model. Besides, it isn’t a great idea to have various factors of comparable data. You can utilize the Pearson connection framework to recognize the factors with high correlation. Furthermore, select one of them utilizing VIF (Variance Inflation Factor). Factors having a higher worth ( VIF > 5 ) can be dropped.

• Principal Component Analysis (PCA)

Karl Pearson has presented this technique. Likewise, it chips away at a condition. That says while the information in a higher-dimensional space needs to map the information in lower measurement space. Albeit, the variance of the data in the lower dimensional space ought to be most extreme. 

It includes two steps: 

• Make the covariance matrix of the dataset. 
• Find the eigenvectors of this matrix. 

We use Eigenvectors relating to the biggest eigenvalues. That is to recreate a huge part of the variance of the original dataset. Henceforth, the number of eigenvectors is less. Also, there may have been some data loss all the while. However, the remaining vectors hold the main variance.

What did we learn?

To sum up this article we have studied dimensionality reduction in a lot of detail. Now, we know why dimensionality reduction in machine learning is an important part of preprocessing the data. Dimensionality is the number of variables, features, or columns which are present in the given dataset. The process of reducing these features is dimensionality reduction. The most professional definition of dimensionality reduction will be stated as: “It is a way of converting the higher dimensions dataset into lesser dimensions dataset ensuring that it provides similar information.”

 

Sources

For more articles, CLICK HERE.