https://codingmachinelearning.wordpress.com/2017/02/13/linear-regression-understanding-the-details/

In  this post, we will see how to do an optimization of parameters for a  simple linear regression. Just  a quick recap:

• Y = MX + C, this it the equation of a straight line which we use to approximate the data.
• M and C are the tunable parameters – we will start with random values of M and C and iteratively converge to an ‘local’ optimal solution.
• The line of best fit is that best parameters M and C, for which the error (squared loss defined in the previous post)  is minimum

Now , I need you to imaging this. When we say (x1,y1), for example (3,6) – this tuple is a point in the xy-space right. Now, what about M and C? M and C are essentially points in the parameter-space and we seek to iteratively converge to best set of M and C for minimizing the error.

Now the next question will be, what is the meaning for iterative convergence ? It essentially means the following sequence of steps:

• Start with random values for M and C
• For that (M,C) – compute loss
• Compute Gradient of the loss – (why on earth do this?)
• Based on the ‘learning rate’ and gradients, update M and C to a more meaningful position in the parameter space.
• And go on and on!

The steps of random values for M and C, and computing loss is quite straight forward. This post, we will deal with gradients and optimization.

Gradients

This is the most important part for learning –  this tells you, in what direction ‘to move’ to get a optimal solution. In plain simple terms, gradients point to the rate of change and points to the direction of steepest ascent for  a given function. If you move in the direction opposite to the gradient, that will help converge to the optimal M and C parameters.

So, we have our cost function and 2 tunable parameters M and C. We have the following gradients to compute:

• Gradient of cost with respect to M
• Gradient of cost with respect to C

Once you compute the gradient of cost with respect to M and C we have a quantifiable value of change in cost, w.r.to a unit change in M and C. Based on this approximation for the direction of descent we update the learnable parameters M and C

Optimization

Once you have the gradients ( direction to move in parameter space – (M,C) space), we now need to update the values of M and C to the place of next meaningful ‘jump’. For that we do the following:

• M = M – learning_rate * gradient of cost w.r.t M
• C = C – learning_rate * gradient of cost w.r.t C

Now you have a new set of values  (M,C), and perform this ‘n’ number of times and Voila you will see convergence.

This method is called gradient descent – which converges to a local optimum. In the next post, we will see what is local optimum, and how to try best to avoid the pitfall of converging to local optimum.

It is very intentional that I have left out the math and the derivative part. I too understand that math puts-off people. I wanted to keep this simple and clear. Do let me know if you are interested in details, I will make a separate post for it. And watch out next week for functions, analysis and more on optimizations!.

In the last post, we saw an English version of linear regression devoid of all mathematics, optimizations blah blah. This post, aims at understanding the mathematical nuances and the beauty of simple gradient descent optimization for obtaining the line of best fit. Just to recap, we have a line of best fit given by linear regression. We claim, that this line captures the trend and pattern in the data. Here is what we are going to do:

• Obtain and understand the data set
• Loading the data set
• Steps in coding the classifier
• Understanding the Bias term in linear regression
• How to define error

Obtain and understand the data set

We will use a simple house price prediction data set – with 2 columns. First column is the area of the house and the second column is the price of the house. In our case, the second column, is the column of interest (also our labels). That means, once I have a trained linear regression model available, given area of a house, I should be able to predict price for it.

Problem:

Given area of the house, we should be able to obtain a prediction price for the same.

Loading the data set

To simplify the loading headache, we will directly use pandas package to help us load the data set (it is plainly a single line code). Pandas will actually load the dataset into memory!! You can refer about pandas here

Steps in coding the classifier

We have the following steps in coding the linear regression classifier: Let X denote the area of the house, and Y denote the house price (labels)

• We have X and Y loaded an ready
• Add a Bias column to X, i.e Add a column of 1’s to X. So our new data matrix will be [colum_of_ones X]
• Define prediction, i.e Y = MX + C, where M is slope and C is intercept (both a floating point numbers!!)
• Define error/cost. We will be using root mean squared error as our cost functions
• Use gradient descent, to find the best values for M and C such that they minimize the error

Don’t worry. We will go through each of these steps in detail. I will also release final version of this code for guidance and checking purposes.

Understanding the Bias term in linear regression

Once we know how to load data, the next step is to add the bias component. Many have this question, why have a bias term? Well I am here to give you an convincing answer. What is our learning objective in linear regression? We need to get the best value for M and C, such that Y = MX + C becomes the line of best fit. Now, see the equation carefully.

Y = M*X + 1*C: This explains the bias term, 1, for each instance.

Say if I don’t have a bias term, and it is zero. Then my equation will be of the form, Y=MX. What is the problem with this? This say that, when X = 0 the Y has to be 0, which imply that the line of best fit has to pass through the origin each time. Got it? This need not always be true, the data need not always be spread around the origin, can be biased. Hence we add the bias term to the network, which can actually help in moving the line of best fit up and down – not constraining it to pass through the origin.

Defining Error in Linear regression

We define error in a very intuitive manner – the simplicity and the ease of it will amaze you. We will:

• Use the difference between the observed (actual) and the predicted value
• Sum the squares of the difference ( squares is a must, otherwise errors may cancel out due to different signs)
• Normalize (divide) by the total number of examples

So the Error Term will be: ∑ Square (| H(x,c) – Y |) / M where

H(x,c) = MX + C and Y is the observed value from our dataset. M is the total number of instances/examples. This is how we define error. Very intuitive right?

Now how do we start the regression? For that you need to wait till the next post that will get you started with random initialization of values for M and C and iteratively converge to the best solution!

Till then, Bye.

To understand big things in life, we need to get small things right. Regression is something very small and trivial as compared to other “big-margin classifier” or Ensemble learning and other buzzwords. This simplicity and the ease of maths is what fascinates me about regression. In this post, we will cover what is the meaning of regression and we will get started with the basics of linear regression.

Regression, in essence is to obtain relationships between variables. In plain terms, it is about finding some pattern or fitting a line/curve that captures this information in the data. Say we have the following image.

How do you describe this image? There is a set of points in the 2-D space and we have obtained a line that somewhat captures the essence of this dataset. For instance, the line shows a linear dependence between x,y coordinates in the dataset. This is as x increases, y value also increases. We can get quite some intuition just by fitting a simple, meaningful line to the data.

The next question would be, will any line work? One cannot draw a random line and claim that it fits the data best. You cannot draw a horizontal line and say, yeah I drew a line that understands the data. This is where we seek the line of best fit for the data set.

The line of best fit problem is in essence to obtain a line that best fits the given data set. How do we know that the line we obtained is best fit or not. Well, we then define goodness of fit as how good the line we have obtained fits/understands the data.

The best line should have minimum error on fitting to the data. Then, next question would probably be, what is the meaning of error?

Error : Estimate of how much each data point deviates from the obtained line

Here in the below image, we can see the line of best fit, The green line from the point to the line shows the deviation/error for that point from the line of best fit. This distance is computed and squared for each point in the dataset and summed to give a good estimate of the error.

I am not going into formulas and derivations in this post, surely the next post will be dealing with that, here my focus is to drive home the point of regression to a common man.

Throughout the post, I have been telling about getting a line, line of best fit blah blah. What exactly is meaning of a line in technical terms. When I say, we get line of best fit, it does not mean that a person sitting inside the computer comes and give a line and fits the data. Line essentially means coefficient. Let me explain.

Line is represented using the equation:

Y = MX + C

Where Y is the target value, M is the slope and C is the intercept term. I will explain these terms later, but to get an understanding of what a line is, see the below table.

When we say we have fit a line, it just means we got the values of M and C, where M=1 and C=0,1 respectively as shown in the above examples.

The process of getting a line of best fit boils down to getting the best set of values for M and C. One solution would be to search every damn point in the euclidean space of points. There are intelligent ways of searching for these coefficients, which we will be seeing in the next post. Till then I will leave you with an hit, “gradient descent algorithm”. Bye

Now we know how the plot looks in 3 dimension. Now we will try to plot the hyperplane for separating this data set

I have manually drawn the hyperplane in the above diagram. We will try to validate our intuition in this case to make sure what I claim actually happens.

Our kernel function is K(x,y)=(x, y, x2+y2 ). Now we need to plot this hyperplane that separate the points.

If we see the values in the plot generated we can see that the value for the hyper plane will be

• X-axis : -1.5 to 1.5  (length of the plane)
• Y-axis : -1.5 to 1.5  (breadth of the plane)
• Z-axis : .5  (height at which the plane separates red and blue)

Plot a meshgrid with these values and we can see that it matches our claim.

The code to generate the plots have been provided in my github account. Here we have manually plotted the decision boundary. Now we will try to get the same result using the scikit learn svm classifier and validate our results to see if it matches our intuition.

This is the output of using  a linear kernel gives accuracy of 58%. We can see its performance is very bad trying to separate non-linear data set. 

The performance of SVM on this data set using a ‘rbf’ kernel is given below. It has 100 percent classification accuracy which is stunning.

This basically is the projection of the hyper-plane on to the lower dimension. But also remember that this does use the gaussian function as its kernel function not what we defined in the beginning.  Please follow the below link to read up more on gaussian kernels.

This wraps up our little discussion on SVM and how it works. In the next post we will look into another classifier – the logistic regression and hope to understand it better.

Code for the Plots available on my github account: https://github.com/vsuriya93/coding-machine-learning/blob/master/SVM/code.py

Useful Links:

1. Gaussian Kernel : https://www.quora.com/What-is-the-intuition-behind-Gaussian-kernel-in-SVM
2. SVM scikit learn: http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC

Link: https://codingmachinelearning.wordpress.com/2016/07/25/support-vector-machines-kernel-explained/

In this post we will continue from where we previously left off. Now we have generated a toy data set looking like a doughnut where your linear decision boundary does not perform well. Look at the data set below

This is how our data set looks. We will define a kernel for this data set and see how this data can be projected up to a 3-dim surface so that the points can be linearly separable.

Kernel is just a function which takes input as an n-dim vector and gives a n+k dimension vector as output, where k is the number of dimension we have projected the data up

In our case we let k=1. So our kernel takes input as 2 dim (x,y) and projects the data to 3 dim (x,y,z)

We define kernel as follows: K(x,y)=(x, y, x2+y2 )

That is, if we have points (x,y) =(1,2) then K(x,y)=(1,2,5) and so on every point in the data set gets transformed. Now if we visualise the data in 3-dim space we have the following transformation of the data.

This is how the kernel transformation works. In this case we have an intuitive idea of what should be the kernel function. But coming with the best kernel for a given problem is close to impossible. So typically in practise choose the Gaussian or RBF Kernel or the polynomial kernel. Remember kernel is just a mathematical function that projects the data up.

Now once this is done, we can see a clear separation of the data set in the higher dimensional plane, so we can apply SVM to get a linear hyperplane that separates the points. Once we have the hyperplane we can project the plane down to a lower dimension and it will look like circle as we saw in the previous post.

Scikit learn provides opportunity to define your own kernel. But this is rarely used. We can use the default option by setting kernel as non-linear or polynomial.

Code for generating the above plots is provided in my github account. The link for the same is given below

Link: https://github.com/vsuriya93/coding-machine-learning/blob/master/SVM/Kernel_transform.py

In the next post we will pick up from here. Now that we have our data projected up – how to get a large margin classifier here? We will be using scikit for building the classifier and plotting the decision boundary.

To summarize:

• Kernels are mathematical functions
• They project data from n-dim to n+k-dim space
• k tells the number of dimensions to project up
• We can manually define kernels – but this is rarely done

References:

• https://www.cs.cmu.edu/~ggordon/SVMs/new-svms-and-kernels.pdf
• http://www.support-vector.net/icml-tutorial.pdf

Link:  https://codingmachinelearning.wordpress.com/2016/07/13/support-vector-machines-svm/

I ended the last post, by saying if we have a not-linearly separable data set then the linear SVM decision boundary will fail miserably !! So what to do then? To help us solve this problem we have SVMs which can give non linear decision boundary. Hurray! We have a solution! However, before that you need to have a question in your mind.

SVM gives hyper-plane as its decision boundary, then how can it be nonlinear ??
Answer to the above question is – you are right. This is what we are going to deal in this post – SVM Kernels

This was not giving us good results. Now how do get a non-linear decision boundary for the above data set? The answer is to use kernels.

Kernel: This actually is a function that maps the data to a higher dimension where the data is separable

In simple words, the data is not linearly separable in this 2-dimension. So I will project the data up by some number of dimensions using a mapping function. In that dimension plane the data will be separable.

Once we get the hyper plane in the n+k dimensional space – where k is the projection of the data up from n dimensional space we can project the hyper-plane back down again to get a decision boundary that can separate points. 

In very simple terms this is what SVM Kernel does. It projects the data up by k dimensions such that the data points are now separable in the higher dimensional plane.

Some popular choice for kernel functions are:

• Gaussian Kernel: https://en.wikipedia.org/wiki/Gaussian_function
• Polynomial Kernel: https://en.wikipedia.org/wiki/Polynomial_kernel

Recap:

• SVMs are powerful in the sense that they can generate non-linear decision boundaries.
• Kernels help in projecting data to a higher dimensional space where the points can be linearly separated.
• Popular ones are RBF (also called the gaussian) and polynomial

In practise we rarely write our own kernel function. For most practical purposes we use the default Gaussian kernel. Nevertheless we can write our own kernel function and see how it works. We will deal with this in our next post. Till then.. Guten Tag!

Reference:

• http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC
• https://www.quora.com/What-are-Kernels-in-Machine-Learning-and-SVM
• http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-034-artificial-intelligence-fall-2010/lecture-videos/lecture-16-learning-support-vector-machines/

Link: https://codingmachinelearning.wordpress.com/2016/07/05/decision-trees-derivation-demystified/

In this post, we deal with one another awesome classifier, called the Support Vector Machines (SVM). They are called large margin classifiers – you will soon understand why. Here is a problem

You see the red points and blue points in the x-y axis of the plot. Our aim is to come up with a margin or a line, which can help us separate the classes (dots). You have 3 choices in front of you. Which one will you choose ? Obviously the dashed line correct? The reason, intuitively your brain has checked the distance between the two classes and fit a line that has the largest margin of divide between the two classes. This is a very basic understanding of the large margin classifier called the SVM. What it does, is to get a hyperplane separating the two classes such that they have a wide margin between them to allow for generalisation.

SVM derivation and proof is quite mathematically taxing, I will deal with it separately later. In this post we will focus on knowing how to use SVM.

First we will create some data points to understand the way SVM works. If the points are linearly separable, then it is just a simple cakewalk. The above diagram explains it all. What if the points are not linearly separable. For example we have data in the form of doughnut – 2 circles, one inside another. Here a simple linear classifier will not work here.

We again use the scikit-learn package to generate the data set for demonstration purposes. Just by using a linear decision boundary we cannot achieve a good classification. Please see the below image for illustration.

What to do in such situation ? how do we classify the points? How to deal with non linearly separable points?

All these questions will be answered in the later posts, but for now it is enough we understand what actually an SVM does.

Now we will move on to how to use SVM. It is quite simple by using scikit. We need to instantiate an object of SVC class and call fit and method and voila we are done.

We will now see how to fit an SVM for data. Say data is in variable train and labels are stored in variable target. We will have the following command to fit an SVM

Simple enough ? Scikit learn package takes care of how to get the best decision boundary to split the points.

To obtain the prediction on the test set invoke the predict method of the classifier on the test set. The prediction will be returned as output.

There you go, you now know how to fit and SVM to the data. It is quite simple when you are using default parameters.

In the next post, we will see about what is the meaning of a kernel, and  how to get an decision boundary that can separate non linearly distributed points and so on.

Also we need to look up on the parameters of SVC() constructor for tuning the parameters etc, which will be explained further in the next set of posts. Till then bye!!

Note: Code for generating the doughnut-shaped dataset is posted on github

https://github.com/vsuriya93/coding-machine-learning/blob/master/decision-tree/create_dataset.py

References:

• http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC
• https://en.wikipedia.org/wiki/Support_vector_machine

https://codingmachinelearning.wordpress.com/2016/06/23/decision-tree-classifier-explanation-example-using-iris-dataset-6/

In this post, we will see how to build a decision tree and a little of its derivation. I will spare you the gory details, but the essence of how we choose a split, what kind of problems we face etc will be dealt in this post.

Say we have the data in the following format: (x1,x2,…,xn) and label y. We have the data this way for instance xi of the data set.Essentially we have a data matrix from which we need to build tree.

How to decide on the node split?

This is one of the most asked questions in decision tree classifiers. How do we decide on which attribute are we going to split the data. In other words, which attribute gives the best split to the data. Splitting randomly may not always give good results. So we have various spitting criterions which we can choose from, say for example, gini,entropy etc

In this post we will deal with gini index as our splitting condition and proceed to describe the process in detail.

Gini Index formula : 1 – [p(j|t)]2

Where p(j|t) refers to the relative frequency of class j at node t

This is a classic example of gini index computation. C1 and C2 are the 2 classes under consideration. We are looking at problems which have 2-way binary split. Gini impurity is a measure of how often a randomly chosen element from the set would be incorrectly labelled if it was randomly labelled according to the distribution of labels in the subset

Now we will do an example for the same, by hand and build a tree manually to get the hang of what happens. But before that we will see the steps in decision tree construction.

• Step 1: Find the best splitting criterion using suitable impurity measures
• Step 2: Partition the data into DataLeft and DataRight based on the attribute splitted
• Repeat Step 1 and Step 2 till all the attributes are covered,

If we keep splitting till the very last attribute, we may be overfitting the data. So we have 2 choices:

1. Pre-Pruning
2. Post-Pruning

Pruning refers to cutting down (chopping down) unwanted branches in our decision tree due to overfitting concerns. Questions like How to select how many branches to prune, should the pruning be done after the construction of tree, or while constructing may arise, but they are out of scope for this blog. I will point you to the places where you can get answers for these questions.

Take the following table and let us find the best split for our decision tree. As said earlier, we will be using gini index as the measure of impurity. We will draw a small table which does the computation for the same

Gini Index – Attribute chosen is body temperature

Gini computation for binary attribute will be as follows.

Warm-blooded:  1 – (⅘)2 – (⅕)2    = 0.32
Cold-blooded:    1 – (0/5)2 – (5/5)2 = 0

Weighted gini index (using class weights – cold,warm blood = 5 each) will be: (5/10) * 0.32 + (5/10) * 0 = .16

Once we are clear on this concept, we will try one more just to become familiar with the idea. This time we will split based on the attribute – Gives birth

We will have the following table if we fill in the values.

Gini computation for binary attribute will be as follows.

Gives Birth Yes :  1 – (3/4)2 – (1/4)2    = 0.375
Gives Birth No:    1 – (1/6)2 – (5/6)2 = 0.28

Weighted gini index (using class weights) will be: (4/10) * 0.375 + (6/10) * 0.280 ~ .318

Between .16 and .318, we choose the attribute with minimum gini impurity – Body temperature. Once we choose this attribute, then remove that columns from the data set. Partition the data into where the classes are yes and no – and continue this process on the left sub-data and right sub-data.

The range of values that Gini, entropy and misclassification error metrics can take, is shown in the above diagram. 

There is no significant reason as to why we choose one over another, just values, but in essence it measures impurity.

We have a fair idea of what is Gini index, entropy is just same but for the formula of computation.

In the next post, we will see about SVM, what it does and how it works. What are kernels and their use etc.

Till then, bye from Suriya.

References:

1. https://www-users.cs.umn.edu/~kumar/dmbook/ch4.pdf   –  Decision tree explained
2. http://scikit-learn.org/stable/modules/tree.html – Implementation aspect of Decision Trees
3. https://en.wikipedia.org/wiki/Decision_tree_learning – What and how of Decision Trees

In this post we will deal with Decision Tree Classifier(DTC). DTC can be simply assumed as a series of if-else questions asked at each decision node. For example if I need to know a given object is a car or not, I will ask a series of questions which can be interpreted as if-else statements. I ask, if the object had 4 tyres. If yes then does it have a box-like structure around it. If yes then does it have a steering-wheel and gear box. If answer to all of these decision questions is yes probably the object is car. Else it may not be a car.

This is a very small example of a decision tree. This is used only for illustration purpose and I hope it drives the message that DTC is nothing but a series of if-else situations at each decision node.

Now we will look into the iris data set and try to implement decision tree classifier algorithm on the same. By implement, most the classifiers have been coded and integrated in scikit sklearn package. We will all we need by using sklearn

Iris data set contains details about different flowers. Based on the features we need to be able to predict the flower type. Please find the description of iris data set here

• First step is to load the iris data set into variables x and y where x contains the data (4 columns) and y contains the target.
• Split the data into train and validation set so that we can see the performance of our model on “previously unseen” instances
• Sepal length,Sepal width, Petal length, Petal width are its features and the targets are setosa (0), Iris virginica(1) and Iris versicolor(2)
• Visualize the data set to see how the points are spread in space

This is the plot we obtain by plotting the first 2 feature points (of sepal length and width). Now we invoke sklearn decision tree classifier to learn from iris data. This simply translates to the following code.

This piece of code, creates an instance of Decision tree classifier and fit method does the fitting of the decision tree. We then invoke predict method to be able to get the prediction for the test-set where our accuracy is close to 90% (on the 30 test instances)

We can also aim to plot the decision surface for the iris data set and see how decision tree classifiers learn rectangular boundaries. The pair wise plots for the iris data set is clearly provided in the sklearn. Link provided at the end of the article.

Our aim here is to perform PCA on the data set, project data on its principle components and reduce data to 2 dimensions. The visualisation of the above task is given below and the code for the same will be made freely available on github.

This is the PCA plot of the data and that now we can focus on obtaining the decision surface of the classifier we have built. Also we can get the tree-type structure for the same using the dot image from sklearn.

We now plot the decision surface for the same.

We can see clearly the rectangular decision boundary learned by our classifier. If a point falls in the blue surface it will be classified as setosa(0) and so on.

Things to remember:

• Decision tree gives rectangular decision boundaries
• It can be thought of as series of if-else questions at each decision node

In the next post we will look into learning SVM and the kind of decision boundary it can generate. Till then bye!!

Code : https://github.com/vsuriya93/coding-machine-learning/tree/master/decision-tree
Sklearn iris visualization: http://scikit-learn.org/stable/auto_examples/tree/plot_iris.html

This post, we will see how to use SVD for reducing dimension. Before that we need to understand the problems associated with PCA so as to appreciate SVD. PCA computes eigenvalues to capture variance. But one cannot compute eigenvalues for non-square matrix.

In SVD we do not have eigenvectors. Instead we have left and right singular vectors and SVD is possible even on non-square matrices. The only bottleneck is the fact that it is a quadratic time algorithm.

The latest thing in NLP is using word vectors for representing word such that the semantics and the context of the word is preserved. The notion of inner product in the vector space triggers the idea of similarity. But if we use one-hot encoding method, computing inner product does not represent similarity. In one hot encoding we represent each word as a vector of 0’s and 1’s where 1 occurs when the word is considered.

For example we have the following 5 words in the dictionary/vocabulary: I, like, motel, hotel, enjoy

The one-hot representation for like will be [0,1,0,0,0] and for enjoy will be [0,0,0,0,1] and so on. Here if we take inner product between hotel and motel which is [0,0,0,1,0] * [0,0,1,0,0] it will tell they are not similar but in reality they are. How to capture this notion. Though we may be solving this problem here, we will use SVD to aid us in this process.

Consider the following 3 sentences in our vocabulary:

1. I like deep learning.
2. I like NLP.
3. I enjoy flying.

We can build a co-occurrence matrix (by taking 1 window context around each word) and to reduce dimension and to capture the word similarity we will use SVD.Code for computing SVD and displaying the plot is also provided in this post.

The “numpy svd” function returns 3 outputs. U, s and v. U contains left singular vectors(need not be a square matrix), s is matrix that contains eigenvalues and v contains the right singular vectors (again, need not be square)

We can decompose the co-occurrence matrix as x=u*s*v . Based on the eigenvalues in s matrix we can reduce dimension by taking k most significant vectors. Thus we can see how SVD can also be used as dimensionality reduction tool.

When we visualise the word vectors we obtained by taking svd of the co-occurrence matrix we get the following output.

We can see that enjoy is closed to like than flying and so on. The dependency is preserved just by choosing the left singular vector and plotting them.

Summary:

• We saw a cool application of SVD in word vector representation.
• SVD is not restricted to square matrices and gives out left and right singular vectors than just eigenvectors
• Using the eigenvalues in the s matrix of u,s,v decomposition and thresholding based on the user’s need we can select first k singular vectors which helps in dimensionality reduction
• SVD is quadratic time algorithm O(N2). On large matrices computational time will increase drastically

Next Post:

In the next series of posts I will be writing about classifiers in general. That is what are the various classifiers we have, and how they behave, what type of decision boundary they represent and so on.

Code for executing the above application is made publicly available on github.Please follow the below link to the code and to the visualization output. Thank you.

Link: https://github.com/vsuriya93/coding-machine-learning/tree/master/SVD