# 程序代写代做代考 python algorithm Microsoft Word – QBUS6850_ass01_StudentVersion.docx

Microsoft Word – QBUS6850_ass01_StudentVersion.docx

Level 7, 263 Clarence St, Sydney, NSW 2000
Tel: 02 9696 7007
E-mail: admin@dingdian.com.au

QBUS 6850 Machine Learning in Business
# 这节课以讲解第一个 individual Assignment 为目的，稍带复习题目相关知识点
# 更多课程知识点梳理和总结请关注期中复习课
# coding时遇到解决不了的报错，善用 Stack Overflow 和官网 documents
# 为防代码一致，请改一下自定义的变量名字，图表名称，以及某些语句

Requirements:
1. Value: 10%
2. Due date: 03/09/2018 (Mon), 17:00
3. Submit WORD document with full explanation and interpretation of any results you obtain
4. Python code in appendix
5. 3 decimal point
6. 10 page limit exluding appendix

# import basic packages

import numpy as np

import pandas as pd

import matplotlib.pyplot as plt

Question-1 Linear Regression (50 marks; week2)
You will work on the UCI ML housing dataset
# for q(a)

from sklearn.datasets import load_boston

from mpl_toolkits.mplot3d import Axes3D

# for q(b)&(C)

from sklearn.linear_model import LinearRegression as lr

from sklearn.cross_validation import train_test_split

a) Suppose you are interested in using the house age AGE (proportion of owner- occupied units
built prior to 1940) as the first feature x1 and the full-value property-tax rate TAX as the
second feature x2 to predict the MEDV (median value of owner-occupied homes in \$1000’s)
as the target t. Write code to extract these two features and the target from the dataset. Use
the dataset (two chosen features and one target) to plot the loss function

That is, we are using a linear regression model without the intercept term β0.
Hint: This is a 3D plot and you will need to iterate over a range of β1 and β2 values.

Level 7, 263 Clarence St, Sydney, NSW 2000
Tel: 02 9696 7007
E-mail: admin@dingdian.com.au

a1. Concept
Loss function
to measure the error between the observed and the model. Loss function, also called a cost
function, which is a single, overall measure of loss incurred in taking any of the available
decisions or actions.

to find the model parameters, we would use gradient descent.

1. We have some random starting point for β1;
2. Keep updating β1 to decrease the loss function L(β1) value;
3. Repeat until achieving minimum (convergence).
4. α (> 0) is called the learning rate: in empirical study, we can try many α values, and

select the one generates least L(β1)
5. gradient descent can converge to a local minima

a2. Code
“””

3D plot

“””

fig = plt.figure()

ax = fig.gca(projection=’3d’)

surf = ax.plot_surface(X, Y, Z,linewidth=0)

Level 7, 263 Clarence St, Sydney, NSW 2000
Tel: 02 9696 7007
E-mail: admin@dingdian.com.au

plt.title(‘Loss function’)

ax.set_xlabel(‘beta_0’)

ax.set_ylabel(‘beta_1’)

ax.set_zlabel(‘Loss Function’)

plt.show()

b) Use the linear regression model LinearRegression in the scikit-learn package to do

two linear regression models to predict the target, with and without the intercept term. You
may use 90% of the data as your training data, and the remaining 10% as your testing data.
Compare the performance of two models and explain the importance of the intercept term.

Hint: The argument fit_intercept of the LinearRegression controls whether an

intercept term is included in the model by fit_intercept = True or

fit_intercept = False.

b2. Code
# Create the linear regression object

lr_obj = LinearRegression()

# Estiamte coefficients

lr_obj.fit(x_data, y_data)

print(“
The estimated model parameters are”)

print(lr_obj.intercept_[0]) # Intercept or eta_0

print(lr_obj.coef_[0,0]) # eta_1

c) Take 90% of data as training data. Construct the centred training dataset by conducting the

following steps in your Python code:
(i) Take the mean of all the training target values, then deduct this mean from  each training

target value MEDV. Take the resulting target values as the new  training target values
𝐭new

(ii) In the training data, take the mean of all the first feature values AGE, then  deduct this
mean from each of first feature values. Take the result as the new first feature values x1new

(iii) In the training data, do the same for the second feature TAX. The result is x2new;

Now build linear regressions with and without the intercept to fit to the new training data.
Report and compare the coefficients and the intercept. Compare the performance of two
models over the testing data. Note that, when you take your testing data into the model to
calculate performance scores, you shall take the relevant training means from the testing
features and targets.

Level 7, 263 Clarence St, Sydney, NSW 2000
Tel: 02 9696 7007
E-mail: admin@dingdian.com.au

d) Consider the closed-form solution of the linear regression below,

see slide 25 (the number may change) of Lecture 2

where X is the design (data) matrix whose first column is all 1s, and the first component in β
is the intercept. Suppose that the data are centred (refer to (c)). Now prove that, in the case of
centred data, the intercept β0 in the solution above is zero.
Hint: You may need that following fact that

where both matrices A and B are invertible.
d1. normal equation

Level 7, 263 Clarence St, Sydney, NSW 2000
Tel: 02 9696 7007
E-mail: admin@dingdian.com.au

Question-2 Logistic Regression (50 marks; week 3)
# for q(a)

from sklearn.linear_model import LogisticRegression

from sklearn.metrics import confusion_matrix, classification_report

Use Logistic Regression to predict diagnosis of breast cancer patients on the Breast Cancer
Wisconsin (Diagnostic) Dataset (wdbc.data). See Section About Datasets. This question aims
to test your ability in programming in matrix operation for Logistic Regression.
a) Write Python code to load the data into your program. For the target feature Diagnosis, change

its literal M (malignant) to 0 and B (benign) to 1. Split the data into training and validation
sets (80%, 20% split). Then define and train a logistic regression model by using scikit-learn’s
LogisticRegression model.
a1. Concept
• logistic function (sigmoid) + regression
• a classification method
• generalized linear model

a2. Code
# logistic regression

lr = LogisticRegression()

lr.fit(X_train, y_train)

y_pred = lr.predict(X_val)

Level 7, 263 Clarence St, Sydney, NSW 2000
Tel: 02 9696 7007
E-mail: admin@dingdian.com.au

b) Using the logistic regression model function below and the

estimated parameters from your model, calculate the probability of sample ID 8510426 (20th
sample)  having a benign diagnosis.

c) The objective of logistic regression is defined as, on slide 17 (the number may change) of

Lecture 3,

where both the parameter β = (β1, β2, …, βd)

T and sample xn = (xn0, xn1, …, xnd)
T are d+1

dimensional vectors, where the intercept feature xn0 = 1. For Wisconsin Dataset d = 30. It is
easy to prove that (you don’t need to prove this)

Write your own python code to use this derivative formula to implement the gradient descent
algorithm for the logistic regression. You may write a python function named such as
myLogisticGD, which accepts an data matrix X, an initial parameter beta_0, and a number
of GD iterations T and other arguments you see appropriate. Your function should return the
learned parameter β.
Hint: In python, you can use the following way to get the vector . First define the
sigmoid function by

def sigmoid(x):

return (1 / (1 + np.exp(-x)))
then

F = sigmoid(np.multiply(X, beta))

c1. Concept

c2. Code
“””

Build the gradient descent function

“””

Level 7, 263 Clarence St, Sydney, NSW 2000
Tel: 02 9696 7007
E-mail: admin@dingdian.com.au

# m denotes the number of training examples here

, not the number of features

def Gradient_Descent_Algo(x, y, beta, alpha, m, numIterations):

xTrans = x.transpose()

for i in range(0, numIterations):

# predicted values from the model

model_0 = np.dot(x, beta)

loss_temp = model_0 – y

# calculte the loss function

loss = np.sum(np.square(loss_temp)) / (2 * m)

# save all the loss function values at each step

loss_total[i]= loss

print(“Iteration: {0} | Loss fucntion: {1}”.format(i, loss))

# calcualte the gradient using matrix representation

gradient = np.dot(xTrans, loss_temp) / m

# update the parameters simulteneously with learning rate

beta = beta – alpha * gradient

# save all the estimated parametes at each step

beta_total[i,:]= beta.transpose()

return beta

d) Based on task (c) and the training data used in (a), write python code to use different initial

values β = (0, 0,…,0)T, β = (1, 1,…,1)T, and a random initial β to start the gradient descent
algorithm to minimise the objective of logistic regression with respect to the parameter β. You
set the number of iteration T=200. Use each resulting β to re-do task (b). Compare the results
and explain the major reasons why you may have different answers with different initial value
for β.

Posted in Uncategorized