A python (numpy) implementation of linear / logistic regression models, optimized with Gradient Descent / Newton's method. An comparison with optimal solution (solved by CVXPY) is also provided.

Install the following packages via pip or conda:

• numpy
• sklearn
• cvxpy
• pickle

Note that I use MOSEK solver for cvxpy (see download link for more detail, and you also need a license), you can also use the default solver (e.g., CVXOPT).

In plot.py, I use latex features for rendering axis labels. To enable this, you need to download latex (see Instruction).

In this task I consider the two convex problems, linear regression and logistic regression.

A classical regression model. Check models/linear_regression.py for more details.

A classical classification model. Check models/logistic_regression.py for more details. Note that the objective is augmented with an L2 norm (otherwise cvx cannot solve it / cannot solve it efficiently).

• In this code, I use MNIST (784 features) for logistic regression (and pick 2 labels for binary classification), and California Housing dataset (8 features)for linear regression.
• Datasets are stored in the folder datasets/.
• The detailed preprocessing of datasets is available in datasets/data_preprocess.py.

In scripts/, I provide the following basic commands:

# linear regression + gradient descent
bash linreg_GD.sh

# linear regression + Newton's method
bash linreg_Newton.sh

# logistic regression + gradient descent
bash logreg_GD.sh

# logistic regression + Newton's method
bash logreg_Newton.sh

Check the results in results/.