【Python机器学习】实验03 逻辑回归-Toy模板网

这篇具有很好参考价值的文章主要介绍了【Python机器学习】实验03 逻辑回归。希望对大家有所帮助。如果存在错误或未考虑完全的地方，请大家不吝赐教，您也可以点击"举报违法"按钮提交疑问。

简单分类模型 - 逻辑回归

在这一次练习中，我们将要实现逻辑回归并且应用到一个分类任务。我们还将通过将正则化加入训练算法，来提高算法的鲁棒性，并用更复杂的情形来测试它。

1.1 准备数据

本实验的数据包含两个变量(评分1和评分2，可以看作是特征),某大学的管理者，想通过申请学生两次测试的评分，来决定他们是否被录取。因此，构建一个可以基于两次测试评分来评估录取可能性的分类模型。

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

#利用pandas显示数据
path = 'ex2data1.txt'
data = pd.read_csv(path, header=None, names=['Exam1', 'Exam2', 'Admitted'])
data.head()

	Exam1	Exam2	Admitted
0	34.623660	78.024693	0
1	30.286711	43.894998	0
2	35.847409	72.902198	0
3	60.182599	86.308552	1
4	79.032736	75.344376	1

data.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 100 entries, 0 to 99
Data columns (total 3 columns):
 #   Column    Non-Null Count  Dtype  
---  ------    --------------  -----  
 0   Exam1     100 non-null    float64
 1   Exam2     100 non-null    float64
 2   Admitted  100 non-null    int64  
dtypes: float64(2), int64(1)
memory usage: 2.5 KB

#看看数据的形状
data.shape

(100, 3)

让我们创建两个分数的散点图，并使用颜色编码来可视化，如果样本是正的（被接纳）或负的（未被接纳）。

positive_index=data["Admitted"].isin([1])
negative_index=data["Admitted"].isin([0])

positive_index

0     False
1     False
2     False
3      True
4      True
      ...  
95     True
96     True
97     True
98     True
99     True
Name: Admitted, Length: 100, dtype: bool

plt.scatter(data[positive_index]["Exam1"],data[positive_index]["Exam2"],color="red",marker="+")
plt.scatter(data[negative_index]["Exam1"],data[negative_index]["Exam2"],color="blue",marker="o")
plt.legend(["admitted","Not admitted"])
plt.xlabel("Exam1")
plt.ylabel("Exam2")
plt.show()

【Python机器学习】实验03 逻辑回归,《机器学习》,机器学习,python,回归

positive = data[data['Admitted'].isin([1])]
negative = data[data['Admitted'].isin([0])]

fig, ax = plt.subplots(figsize=(6,4))
ax.scatter(positive['Exam1'],
           positive['Exam2'],
           s=50,
           c='b',
           marker='o',
           label='Admitted')
ax.scatter(negative['Exam1'],
           negative['Exam2'],
           s=50,
           c='r',
           marker='x',
           label='Not Admitted')
ax.legend()
ax.set_xlabel('Exam 1 Score')
ax.set_ylabel('Exam 2 Score')
plt.show()

【Python机器学习】实验03 逻辑回归,《机器学习》,机器学习,python,回归

看起来在两类间，有一个清晰的决策边界。现在我们需要实现逻辑回归，那样就可以训练一个模型来预测结果。

#准备训练数据
col_num=data.shape[1]
X=data.iloc[:,:col_num-1]
y=data.iloc[:,col_num-1]

X.insert(0,"ones",1)
X.shape

(100, 3)

X=X.values
X.shape

(100, 3)

y=y.values
y.shape

(100,)

1.2 定义假设函数

Sigmoid 函数

$g$ 代表一个常用的逻辑函数（logistic function）为 $S$ 形函数（Sigmoid function），公式为：
$g(z)=\frac{1}{1+{{e}^{-z}}}$
合起来，我们得到逻辑回归模型的假设函数：
${{h}}\left( x \right)=\frac{1}{1+{{e}^{-{{w }^{T}}x}}}$

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

让我们做一个快速的检查，来确保它可以工作。

nums = np.arange(-10, 10, step=1)
fig, ax = plt.subplots(figsize=(6, 4))
ax.plot(nums, sigmoid(nums), 'r')
plt.show()

【Python机器学习】实验03 逻辑回归,《机器学习》,机器学习,python,回归

w=np.zeros((X.shape[1],1))

#定义假设函数h(x)=1/(1+exp^(-w.Tx))
def h(X,w):
    z=X@w
    h=sigmoid(z)
    return h

1.3 定义代价函数

y_hat=sigmoid(X@w)

X.shape,y.shape,np.log(y_hat).shape

((100, 3), (100,), (100, 1))

现在，我们需要编写代价函数来评估结果。
代价函数：
$J\left(w\right)=-\frac{1}{m}\sum\limits_{i=1}^{m}{({{y}^{(i)}}\log \left( {h}\left( {{x}^{(i)}} \right) \right)+\left( 1-{{y}^{(i)}} \right)\log \left( 1-{h}\left( {{x}^{(i)}} \right) \right))}$

#代价函数构造
def cost(X,w,y):
    #当X(m,n+1),y(m,),w(n+1,1)
    y_hat=sigmoid(X@w)
    right=np.multiply(y.ravel(),np.log(y_hat).ravel())+np.multiply((1-y).ravel(),np.log(1-y_hat).ravel())
    cost=-np.sum(right)/X.shape[0]
    return cost

#设置初始的权值
w=np.zeros((X.shape[1],1))
#查看初始的代价
cost(X,w,y)

0.6931471805599453

看起来不错，接下来，我们需要一个函数来计算我们的训练数据、标签和一些参数 $w$ 的梯度。

1.4 定义梯度下降算法

gradient descent(梯度下降)

这是批量梯度下降（batch gradient descent）
转化为向量化计算： $\frac{1}{m} X^T( Sigmoid(XW) - y )$
$\frac{\partial J\left( w \right)}{\partial {{w }_{j}}}=\frac{1}{m}\sum\limits_{i=1}^{m}{({{h}}\left( {{x}^{(i)}} \right)-{{y}^{(i)}})x_{_{j}}^{(i)}}$

def grandient(X,y,iter_num,alpha):
    y=y.reshape((X.shape[0],1))
    w=np.zeros((X.shape[1],1))
    cost_lst=[]
   
    for i in range(iter_num):
        y_pred=h(X,w)-y
        temp=np.zeros((X.shape[1],1))
        for j in range(X.shape[1]):
            right=np.multiply(y_pred.ravel(),X[:,j])
            
            gradient=1/(X.shape[0])*(np.sum(right))
            temp[j,0]=w[j,0]-alpha*gradient
        w=temp
        cost_lst.append(cost(X,w,y.ravel()))
    return w,cost_lst

iter_num,alpha=1000000,0.001
w,cost_lst=grandient(X,y,iter_num,alpha)

cost_lst[iter_num-1]

0.22465416189188264

plt.plot(range(iter_num),cost_lst,"b-o")

[<matplotlib.lines.Line2D at 0x14224c08190>]

【Python机器学习】实验03 逻辑回归,《机器学习》,机器学习,python,回归

Xw—X(m,n) w (n,1)

array([[-15.39517866],
       [  0.12825989],
       [  0.12247929]])

1.5 绘制决策边界

0=w[0,0]+w[1,0]*x1+w[2,0]*x2,令y=0 可以得到x2和x1的关系为
x2=(-w[0,0]-w[1,0]*x1)/w[2,0]

#绘图
x_exma1=np.linspace(data["Exam1"].min(),data["Exam1"].max(),100)
x2=(-w[0,0]-w[1,0]*x_exma1)/(w[2,0])
plt.plot(x_exma1,x2,"r-")
plt.scatter(data[positive_index]["Exam1"],data[positive_index]["Exam2"],color="c",marker="^")
plt.scatter(data[negative_index]["Exam1"],data[negative_index]["Exam2"],color="b",marker="o")
plt.show()

【Python机器学习】实验03 逻辑回归,《机器学习》,机器学习,python,回归

1.6 计算准确率

如何用我们所学的参数w来为数据集X输出预测，来给我们的分类器的训练精度打分。
逻辑回归模型的假设函数：
${{h}}\left( x \right)=\frac{1}{1+{{e}^{-{{w }^{T}}X}}}$

当 ${{h}}$ 大于等于0.5时，预测 y=1

当 ${{h}}$ 小于0.5时，预测 y=0 。

y_p_true=(h(X,w)>0.5).ravel()
y_p_true

array([False, False, False,  True,  True, False,  True, False,  True,
        True,  True, False,  True,  True, False,  True, False, False,
        True,  True, False,  True, False, False,  True,  True,  True,
        True, False, False,  True,  True, False, False, False, False,
        True,  True, False, False,  True, False,  True,  True, False,
       False,  True,  True,  True,  True,  True,  True,  True, False,
       False, False,  True,  True,  True,  True,  True, False, False,
       False, False, False,  True, False,  True,  True, False,  True,
        True,  True,  True,  True,  True,  True, False,  True,  True,
        True,  True, False,  True,  True, False,  True,  True, False,
        True,  True, False,  True,  True,  True,  True,  True, False,
        True])

y_p_pred=(data["Admitted"]==1).values
y_p_pred

array([False, False, False,  True,  True, False,  True,  True,  True,
        True, False, False,  True,  True, False,  True,  True, False,
        True,  True, False,  True, False, False,  True,  True,  True,
       False, False, False,  True,  True, False,  True, False, False,
       False,  True, False, False,  True, False,  True, False, False,
       False,  True,  True,  True,  True,  True,  True,  True, False,
       False, False,  True, False,  True,  True,  True, False, False,
       False, False, False,  True, False,  True,  True, False,  True,
        True,  True,  True,  True,  True,  True, False, False,  True,
        True,  True,  True,  True,  True, False,  True,  True, False,
        True,  True, False,  True,  True,  True,  True,  True,  True,
        True])

np.sum(y_p_pred==y_p_true)/X.shape[0]

0.89

1.7 试试用Sklearn来解决

from sklearn.linear_model import LogisticRegression
clf = LogisticRegression().fit(X, y)
clf.score(X,y)

0.89

clf.predict(X)

array([0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1,
       0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1,
       0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0,
       1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0, 1, 1, 0, 1,
       1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1], dtype=int64)

np.array([1 if item>0.5 else 0 for item in h(X,w)])

array([0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1,
       0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1,
       0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0,
       1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0, 1, 1, 0, 1,
       1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1])

np.argmax(clf.predict_proba(X),axis=1)

array([0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1,
       0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1,
       0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0,
       1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0, 1, 1, 0, 1,
       1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1], dtype=int64)

X.shape,y.shape

((100, 3), (100,))

from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
y
clf = LogisticRegression().fit(X, y)
clf.predict(X)

array([0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1,
       0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1,
       0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0,
       1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0, 1, 1, 0, 1,
       1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1], dtype=int64)

clf.predict(X).shape

(100,)

y.shape

(100,)

np.sum(clf.predict(X)==y.ravel())/np.sum(X.shape[0])

0.89

#所以分类问题中的score用的是准确率
clf.score(X,y)

0.89

我们的逻辑回归分类器预测正确，如果一个学生被录取或没有录取，达到89%的精确度。不坏！记住，这是训练集的准确性。我们没有保持住了设置或使用交叉验证得到的真实逼近，所以这个数字有可能高于其真实值（这个话题将在以后说明）。

2.1 准备数据(试试第二个例子)

在训练的第二部分，我们将要通过加入正则项提升逻辑回归算。简而言之，正则化是成本函数中的一个术语，它使算法更倾向于“更简单”的模型（在这种情况下，模型将更小的系数）。这个理论助于减少过拟合，提高模型的泛化能力。

设想你是工厂的生产主管，你有一些芯片在两次测试中的测试结果。对于这两次测试，你想决定是否芯片要被接受或抛弃。为了帮助你做出艰难的决定，你拥有过去芯片的测试数据集，从其中你可以构建一个逻辑回归模型。

和第一部分很像，从数据可视化开始吧！

#读取文件'ex2data2.txt'的数据
path="ex2data2.txt"
data2=pd.read_csv(path,header=None,names=["Test1","Test2","Accepted"])
data2.head()

	Test1	Test2	Accepted
0	0.051267	0.69956	1
1	-0.092742	0.68494	1
2	-0.213710	0.69225	1
3	-0.375000	0.50219	1
4	-0.513250	0.46564	1

#可视化数据
positive_index=data2["Accepted"]==1
negative_index=data2["Accepted"]==0
plt.scatter(data2[positive_index]["Test1"],data2[positive_index]["Test2"],color="r",marker="^")
plt.scatter(data2[negative_index]["Test1"],data2[negative_index]["Test2"],color="b",marker="o")
plt.legend(["Accpted","Not accepted"])
plt.show()

【Python机器学习】实验03 逻辑回归,《机器学习》,机器学习,python,回归

X2=data2.iloc[:,:2]
y2=data2.iloc[:,2]
X2.insert(0,"ones",1)
X2.shape,y2.shape

((118, 3), (118,))

X2=X2.values
y2=y2.values

2.2 假设函数与前h相同

2.3 代价函数与前相同

2.4 梯度下降算法与前相同

iter_num,alpha=600000,0.0005
w,cost_lst=grandient(X2,y2,iter_num,alpha)

#绘制误差曲线
plt.plot(range(iter_num),cost_lst,"b-o")

[<matplotlib.lines.Line2D at 0x1422d45e970>]

【Python机器学习】实验03 逻辑回归,《机器学习》,机器学习,python,回归

#看看准确率有多少
y_pred=[1 if item>=0.5 else 0  for item in sigmoid(X2@w).ravel()]
y_pred=np.array(y_pred)
y_pred.shape

(118,)

y2.shape

(118,)

np.sum(y_pred==y2)

np.sum(y_pred==y2)/y2.shape[0]

0.5508474576271186

y_pred=[1 if item>=0.5 else 0  for item in sigmoid(X2@w).ravel()]
y_pred=np.array(y_pred)
np.sum(y_pred==y2)/y2.shape[0]

0.5508474576271186

2.5 欠拟合的了（模型过于简单，增加一些多项式特征）

path="ex2data2.txt"
data2=pd.read_csv(path,header=None,names=["Test1","Test2","Accepted"])
data2.head()

	Test1	Test2	Accepted
0	0.051267	0.69956	1
1	-0.092742	0.68494	1
2	-0.213710	0.69225	1
3	-0.375000	0.50219	1
4	-0.513250	0.46564	1

#为数据框增加多列多项式特征
def poly_feature(data2,degree):
    x1=data2["Test1"]
    x2=data2["Test2"]
    items=[]
    for i in range(degree+1):
        for j in range(degree-i+1):
            data2["F"+str(i)+str(j)]=np.power(x1,i)*np.power(x2,j)
            items.append("(x1**{})*(x2**{})".format(i,j))
    data2=data2.drop(["Test1","Test2"],axis=1)
    return data2,items
data2,items=poly_feature(data2,4)

data2.shape

(118, 16)

data2.head(5)

	Accepted	F00	F01	F02	F03	F04	F10	F11	F12	F13	F20	F21	F22	F30	F31	F40
0	1	1.0	0.69956	0.489384	0.342354	0.239497	0.051267	0.035864	0.025089	0.017551	0.002628	0.001839	0.001286	0.000135	0.000094	0.000007
1	1	1.0	0.68494	0.469143	0.321335	0.220095	-0.092742	-0.063523	-0.043509	-0.029801	0.008601	0.005891	0.004035	-0.000798	-0.000546	0.000074
2	1	1.0	0.69225	0.479210	0.331733	0.229642	-0.213710	-0.147941	-0.102412	-0.070895	0.045672	0.031616	0.021886	-0.009761	-0.006757	0.002086
3	1	1.0	0.50219	0.252195	0.126650	0.063602	-0.375000	-0.188321	-0.094573	-0.047494	0.140625	0.070620	0.035465	-0.052734	-0.026483	0.019775
4	1	1.0	0.46564	0.216821	0.100960	0.047011	-0.513250	-0.238990	-0.111283	-0.051818	0.263426	0.122661	0.057116	-0.135203	-0.062956	0.069393

X2=data2.iloc[:,1:data2.shape[1]-1]
y2=data2.iloc[:,0]
X2.shape,y.shape

((118, 14), (100,))

X2

	F00	F01	F02	F03	F04	F10	F11	F12	F13	F20	F21	F22	F30	F31
0	1.0	0.699560	0.489384	0.342354	2.394969e-01	0.051267	0.035864	0.025089	0.017551	0.002628	0.001839	0.001286	1.347453e-04	9.426244e-05
1	1.0	0.684940	0.469143	0.321335	2.200950e-01	-0.092742	-0.063523	-0.043509	-0.029801	0.008601	0.005891	0.004035	-7.976812e-04	-5.463638e-04
2	1.0	0.692250	0.479210	0.331733	2.296423e-01	-0.213710	-0.147941	-0.102412	-0.070895	0.045672	0.031616	0.021886	-9.760555e-03	-6.756745e-03
3	1.0	0.502190	0.252195	0.126650	6.360222e-02	-0.375000	-0.188321	-0.094573	-0.047494	0.140625	0.070620	0.035465	-5.273438e-02	-2.648268e-02
4	1.0	0.465640	0.216821	0.100960	4.701118e-02	-0.513250	-0.238990	-0.111283	-0.051818	0.263426	0.122661	0.057116	-1.352032e-01	-6.295600e-02
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
113	1.0	0.538740	0.290241	0.156364	8.423971e-02	-0.720620	-0.388227	-0.209153	-0.112679	0.519293	0.279764	0.150720	-3.742131e-01	-2.016035e-01
114	1.0	0.494880	0.244906	0.121199	5.997905e-02	-0.593890	-0.293904	-0.145447	-0.071979	0.352705	0.174547	0.086380	-2.094682e-01	-1.036616e-01
115	1.0	0.999270	0.998541	0.997812	9.970832e-01	-0.484450	-0.484096	-0.483743	-0.483390	0.234692	0.234520	0.234349	-1.136964e-01	-1.136134e-01
116	1.0	0.999270	0.998541	0.997812	9.970832e-01	-0.006336	-0.006332	-0.006327	-0.006323	0.000040	0.000040	0.000040	-2.544062e-07	-2.542205e-07
117	1.0	-0.030612	0.000937	-0.000029	8.781462e-07	0.632650	-0.019367	0.000593	-0.000018	0.400246	-0.012252	0.000375	2.532156e-01	-7.751437e-03

118 rows × 14 columns

y2

0      1
1      1
2      1
3      1
4      1
      ..
113    0
114    0
115    0
116    0
117    0
Name: Accepted, Length: 118, dtype: int64

X2=X2.values
y2=y2.values

X2.shape,y2.shape

((118, 14), (118,))

#虽然加了多项式特征，但是其他地方不需要改变
iter_num,alpha=600000,0.001
w,cost_lst=grandient(X2,y2,iter_num,alpha)
w,cost_lst

(array([[ 3.03503577],
        [ 3.20158942],
        [-4.0495866 ],
        [-1.04983379],
        [-3.95636068],
        [ 2.0490215 ],
        [-3.40302089],
        [-0.79821365],
        [-1.23393575],
        [-7.32541507],
        [-1.41115593],
        [-1.80717912],
        [-0.54355034],
        [ 0.11775491]]),
 [0.6931399371004173,
  0.6931326952275558,
  0.6931254549404754,
  0.6931182162382921,
  0.693110979120122,
  0.6931037435850823,
  0.69309650963229,
  0.6930892772608634,
  0.6930820464699207,
  0.6930748172585808,
  0.6930675896259637,
  0.6930603635711893,
  0.6930531390933783,
  0.693045916191652,
  0.6930386948651323,
  0.6930314751129414,
  0.6930242569342021,
  0.6930170403280382,
  0.6930098252935735,
  0.6930026118299326,
  0.6929953999362406,
  0.6929881896116231,
  0.6929809808552065,
  0.6929737736661173,
  0.6929665680434831,
  0.6929593639864312,
  0.6929521614940908,
  0.6929449605655903,
  0.6929377612000593,
  0.6929305633966278,
  0.6929233671544265,
  0.6929161724725863,
  0.6929089793502394,
  0.6929017877865175,
  0.6928945977805535,
  0.6928874093314809,
  0.6928802224384333,
  0.6928730371005455,
  0.6928658533169517,
  0.6928586710867882,
  0.6928514904091909,
  0.6928443112832959,
  0.6928371337082408,
  0.6928299576831629,
  0.6928227832072007,
  0.6928156102794925,
  0.692808438899178,
  0.6928012690653973,
  0.6927941007772903,
  0.6927869340339977,
  0.6927797688346614,
  0.6927726051784232,
  0.6927654430644256,
  0.6927582824918116,
  0.6927511234597252,
  0.6927439659673101,
  0.6927368100137112,
  0.6927296555980735,
  0.6927225027195429,
  0.6927153513772657,
  0.6927082015703889,
  0.6927010532980594,
  0.6926939065594254,
  0.6926867613536353,
  0.6926796176798382,
  0.6926724755371833,
  0.6926653349248207,
  0.692658195841901,
  0.6926510582875753,
  0.6926439222609956,
  0.6926367877613135,
  0.6926296547876825,
  0.6926225233392544,
  0.6926153934151844,
  0.6926082650146258,
  0.6926011381367343,
  0.6925940127806647,
  0.6925868889455729,
  0.6925797666306154,
  0.6925726458349493,
  0.692565526557732,
  0.6925584087981212,
  0.6925512925552758,
  0.6925441778283548,
  0.6925370646165178,
  0.6925299529189249,
  0.6925228427347367,
  0.6925157340631142,
  0.6925086269032195,
  0.6925015212542147,
  0.6924944171152624,
  0.6924873144855257,
  0.692480213364169,
  0.6924731137503559,
  0.692466015643252,
  0.6924589190420221,
  0.6924518239458325,
  0.6924447303538497,
  0.6924376382652399,
  0.6924305476791713,
  0.6924234585948119,
  0.6924163710113299,
  0.6924092849278943,
  0.6924022003436747,
  0.6923951172578418,
  0.6923880356695654,
  0.692380955578017,
  0.6923738769823684,
  0.6923667998817915,
  0.6923597242754587,
  0.6923526501625441,
  0.6923455775422206,
  0.6923385064136628,
  0.6923314367760456,
  0.6923243686285434,
  0.6923173019703337,
  0.6923102368005914,
  0.6923031731184935,
  0.6922961109232177,
  0.6922890502139424,
  0.6922819909898448,
  0.6922749332501046,
  0.692267876993901,
  0.6922608222204141,
  0.692253768928824,
  0.6922467171183119,
  0.6922396667880594,
  0.6922326179372482,
  0.692225570565061,
  0.6922185246706808,
  0.6922114802532913,
  0.6922044373120759,
  0.69219739584622,
  0.692190355854908,
  0.692183317337326,
  0.6921762802926597,
  0.6921692447200962,
  0.6921622106188222,
  0.6921551779880252,
  0.6921481468268937,
  0.6921411171346162,
  0.6921340889103819,
  0.6921270621533807,
  0.6921200368628022,
  0.6921130130378376,
  0.6921059906776778,
  0.6920989697815146,
  0.6920919503485404,
  0.6920849323779475,
  0.6920779158689296,
  0.6920709008206801,
  0.6920638872323933,
  0.6920568751032641,
  0.6920498644324875,
  0.6920428552192593,
  0.692035847462776,
  0.6920288411622341,
  0.6920218363168312,
  0.6920148329257646,
  0.6920078309882332,
  0.692000830503435,
  0.6919938314705699,
  0.6919868338888373,
  0.6919798377574379,
  0.6919728430755722,
  0.6919658498424414,
  0.6919588580572479,
  0.6919518677191929,
  0.6919448788274806,
  0.6919378913813131,
  0.6919309053798947,
  0.6919239208224298,
  0.691916937708123,
  0.6919099560361796,
  0.6919029758058055,
  0.691895997016207,
  0.6918890196665908,
  0.6918820437561641,
  0.6918750692841349,
  0.6918680962497116,
  0.6918611246521027,
  0.6918541544905173,
  0.691847185764166,
  0.6918402184722583,
  0.6918332526140052,
  0.6918262881886181,
  0.6918193251953085,
  0.691812363633289,
  0.691805403501772,
  0.691798444799971,
  0.6917914875270996,
  0.6917845316823721,
  0.6917775772650034,
  0.6917706242742084,
  0.6917636727092031,
  0.6917567225692034,
  0.6917497738534262,
  0.6917428265610885,
  0.6917358806914085,
  0.691728936243604,
  0.6917219932168934,
  0.6917150516104965,
  0.6917081114236324,
  0.6917011726555214,
  0.6916942353053842,
  0.691687299372442,
  0.6916803648559162,
  0.691673431755029,
  0.6916665000690029,
  0.691659569797061,
  0.6916526409384272,
  0.691645713492325,
  0.6916387874579791,
  0.6916318628346149,
  0.6916249396214572,
  0.6916180178177327,
  0.6916110974226677,
  0.6916041784354889,
  0.6915972608554238,
  0.6915903446817003,
  0.6915834299135475,
  0.6915765165501937,
  0.691569604590868,
  0.6915626940348011,
  0.6915557848812225,
  0.6915488771293637,
  0.6915419707784556,
  0.6915350658277303,
  0.69152816227642,
  0.691521260123757,
  0.6915143593689757,
  0.6915074600113087,
  0.6915005620499908,
  0.6914936654842565,
  0.6914867703133412,
  0.6914798765364804,
  0.6914729841529099,
  0.691466093161867,
  0.6914592035625885,
  0.6914523153543117,
  0.6914454285362747,
  0.6914385431077166,
  0.6914316590678757,
  0.6914247764159921,
  0.6914178951513054,
  0.6914110152730563,
  0.6914041367804855,
  0.6913972596728342,
  0.6913903839493448,
  0.6913835096092594,
  0.6913766366518207,
  0.6913697650762726,
  0.6913628948818578,
  0.6913560260678212,
  0.6913491586334077,
  0.6913422925778623,
  0.6913354279004306,
  0.6913285646003587,
  0.6913217026768933,
  0.6913148421292818,
  0.6913079829567716,
  0.6913011251586102,
  0.691294268734047,
  0.6912874136823305,
  0.6912805600027101,
  0.6912737076944359,
  0.6912668567567581,
  0.6912600071889275,
  0.691253158990196,
  0.6912463121598151,
  0.691239466697037,
  0.6912326226011144,
  0.6912257798713006,
  0.6912189385068493,
  0.6912120985070148,
  0.6912052598710514,
  0.6911984225982147,
  0.6911915866877595,
  0.6911847521389427,
  0.69117791895102,
  0.6911710871232487,
  0.6911642566548867,
  0.6911574275451913,
  0.6911505997934211,
  0.6911437733988343,
  0.6911369483606911,
  0.6911301246782512,
  0.6911233023507742,
  0.6911164813775215,
  0.6911096617577531,
  0.691102843490732,
  0.6910960265757193,
  0.6910892110119781,
  0.6910823967987713,
  0.6910755839353621,
  0.6910687724210147,
  0.6910619622549933,
  0.6910551534365628,
  0.6910483459649887,
  0.6910415398395364,
  0.6910347350594729,
  0.6910279316240643,
  0.6910211295325772,
  0.6910143287842805,
  0.6910075293784415,
  0.6910007313143287,
  0.6909939345912114,
  0.6909871392083592,
  0.6909803451650415,
  0.6909735524605287,
  0.6909667610940924,
  0.6909599710650032,
  0.6909531823725328,
  0.6909463950159536,
  0.6909396089945385,
  0.6909328243075601,
  0.6909260409542926,
  0.6909192589340094,
  0.6909124782459855,
  0.6909056988894955,
  0.690898920863815,
  0.6908921441682196,
  0.6908853688019859,
  0.6908785947643907,
  0.6908718220547111,
  0.6908650506722244,
  0.6908582806162092,
  0.6908515118859441,
  0.6908447444807079,
  0.6908379783997801,
  0.6908312136424405,
  0.6908244502079699,
  0.6908176880956487,
  0.6908109273047588,
  0.6908041678345811,
  0.6907974096843985,
  0.6907906528534933,
  0.6907838973411485,
  0.690777143146648,
  0.6907703902692756,
  0.6907636387083155,
  0.6907568884630527,
  0.690750139532773,
  0.690743391916762,
  0.6907366456143056,
  0.6907299006246905,
  0.6907231569472044,
  0.6907164145811341,
  0.6907096735257682,
  0.690702933780395,
  0.6906961953443035,
  0.6906894582167828,
  0.690682722397123,
  0.6906759878846143,
  0.6906692546785472,
  0.6906625227782132,
  0.6906557921829036,
  0.6906490628919102,
  0.6906423349045264,
  0.6906356082200443,
  0.6906288828377575,
  0.6906221587569599,
  0.6906154359769455,
  0.6906087144970098,
  0.6906019943164466,
  0.6905952754345525,
  0.6905885578506235,
  0.6905818415639554,
  0.6905751265738461,
  0.690568412879592,
  0.6905617004804914,
  0.6905549893758427,
  0.6905482795649438,
  0.6905415710470946,
  0.6905348638215943,
  0.6905281578877431,
  0.6905214532448414,
  0.6905147498921895,
  0.6905080478290895,
  0.690501347054843,
  0.6904946475687516,
  0.6904879493701188,
  0.6904812524582472,
  0.6904745568324402,
  0.6904678624920019,
  0.6904611694362365,
  0.6904544776644493,
  0.6904477871759448,
  0.6904410979700292,
  0.6904344100460088,
  0.6904277234031898,
  0.6904210380408787,
  0.6904143539583844,
  0.6904076711550134,
  0.6904009896300746,
  0.6903943093828763,
  0.6903876304127281,
  0.6903809527189395,
  0.6903742763008206,
  0.6903676011576815,
  0.6903609272888331,
  0.6903542546935871,
  0.6903475833712557,
  0.69034091332115,
  0.6903342445425832,
  0.6903275770348686,
  0.6903209107973187,
  0.6903142458292487,
  0.6903075821299722,
  0.6903009196988041,
  0.6902942585350598,
  0.6902875986380544,
  0.6902809400071047,
  0.6902742826415266,
  0.6902676265406373,
  0.6902609717037543,
  0.6902543181301951,
  0.6902476658192781,
  0.6902410147703216,
  0.6902343649826451,
  0.6902277164555679,
  0.6902210691884098,
  0.6902144231804913,
  0.6902077784311333,
  0.690201134939657,
  0.6901944927053835,
  0.6901878517276353,
  0.6901812120057349,
  0.6901745735390051,
  0.6901679363267692,
  0.6901613003683511,
  0.6901546656630752,
  0.6901480322102651,
  0.6901414000092467,
  0.6901347690593452,
  0.6901281393598866,
  0.690121510910197,
  0.6901148837096033,
  0.6901082577574326,
  0.6901016330530122,
  0.6900950095956704,
  0.6900883873847353,
  0.6900817664195361,
  0.6900751466994021,
  0.6900685282236624,
  0.6900619109916476,
  0.6900552950026879,
  0.6900486802561143,
  0.6900420667512582,
  0.6900354544874515,
  0.6900288434640263,
  0.690022233680315,
  0.6900156251356507,
  0.6900090178293673,
  0.6900024117607981,
  0.6899958069292774,
  0.6899892033341402,
  0.6899826009747214,
  0.6899759998503566,
  0.6899693999603816,
  0.6899628013041332,
  0.6899562038809481,
  0.6899496076901628,
  0.6899430127311155,
  0.6899364190031444,
  0.6899298265055872,
  0.6899232352377836,
  0.6899166451990724,
  0.6899100563887932,
  0.6899034688062867,
  0.6898968824508926,
  0.6898902973219526,
  0.6898837134188075,
  0.6898771307407991,
  0.6898705492872698,
  0.6898639690575618,
  0.6898573900510188,
  0.6898508122669835,
  0.6898442357048,
  0.6898376603638127,
  0.689831086243366,
  0.6898245133428047,
  0.6898179416614751,
  0.689811371198722,
  0.6898048019538924,
  0.689798233926333,
  0.6897916671153906,
  0.6897851015204128,
  0.6897785371407474,
  0.6897719739757432,
  0.6897654120247485,
  0.6897588512871126,
  0.6897522917621844,
  0.6897457334493153,
  0.6897391763478544,
  0.689732620457153,
  0.6897260657765623,
  0.6897195123054336,
  0.6897129600431194,
  0.6897064089889716,
  0.6896998591423432,
  0.6896933105025878,
  0.6896867630690583,
  0.6896802168411088,
  0.6896736718180946,
  0.6896671279993697,
  0.6896605853842896,
  0.6896540439722101,
  0.6896475037624867,
  0.6896409647544766,
  0.6896344269475361,
  0.6896278903410226,
  0.689621354934294,
  0.6896148207267083,
  0.6896082877176235,
  0.6896017559063989,
  0.6895952252923941,
  0.689588695874968,
  0.6895821676534813,
  0.6895756406272939,
  0.6895691147957672,
  0.6895625901582623,
  0.6895560667141409,
  0.6895495444627653,
  0.6895430234034974,
  0.6895365035357004,
  0.6895299848587378,
  0.6895234673719729,
  0.6895169510747703,
  0.6895104359664939,
  0.6895039220465085,
  0.6894974093141799,
  0.6894908977688735,
  0.6894843874099555,
  0.6894778782367924,
  0.6894713702487508,
  0.6894648634451982,
  0.6894583578255021,
  0.6894518533890304,
  0.6894453501351522,
  0.6894388480632355,
  0.6894323471726499,
  0.6894258474627657,
  0.6894193489329519,
  0.6894128515825795,
  0.6894063554110188,
  0.6893998604176417,
  0.6893933666018192,
  0.6893868739629238,
  0.6893803825003278,
  0.6893738922134036,
  0.6893674031015249,
  0.689360915164065,
  0.6893544284003981,
  0.6893479428098981,
  0.6893414583919399,
  0.6893349751458988,
  0.6893284930711506,
  0.6893220121670708,
  0.6893155324330359,
  0.6893090538684223,
  0.6893025764726077,
  0.6892961002449691,
  0.6892896251848848,
  0.6892831512917327,
  0.6892766785648914,
  0.68927020700374,
  0.6892637366076584,
  0.6892572673760258,
  0.689250799308223,
  0.6892443324036301,
  0.6892378666616283,
  0.689231402081599,
  0.6892249386629243,
  0.6892184764049858,
  0.6892120153071661,
  0.6892055553688485,
  0.689199096589416,
  0.6891926389682527,
  0.689186182504742,
  0.6891797271982693,
  0.6891732730482184,
  0.6891668200539756,
  0.6891603682149255,
  0.6891539175304552,
  0.6891474679999502,
  0.6891410196227977,
  0.6891345723983848,
  0.6891281263260991,
  0.6891216814053285,
  0.6891152376354615,
  0.6891087950158865,
  0.6891023535459927,
  0.6890959132251698,
  0.6890894740528071,
  0.6890830360282953,
  0.689076599151025,
  0.6890701634203871,
  0.6890637288357729,
  0.6890572953965745,
  0.6890508631021836,
  0.6890444319519929,
  0.6890380019453953,
  0.6890315730817844,
  0.6890251453605535,
  0.6890187187810966,
  0.689012293342808,
  0.6890058690450832,
  0.688999445887317,
  0.6889930238689047,
  0.6889866029892425,
  0.6889801832477265,
  0.6889737646437539,
  0.6889673471767213,
  0.6889609308460263,
  0.6889545156510667,
  0.6889481015912406,
  0.688941688665947,
  0.6889352768745843,
  0.6889288662165526,
  0.6889224566912505,
  0.6889160482980794,
  0.6889096410364388,
  0.6889032349057299,
  0.6888968299053541,
  0.6888904260347121,
  0.688884023293207,
  0.6888776216802407,
  0.6888712211952157,
  0.6888648218375353,
  0.688858423606603,
  0.6888520265018226,
  0.6888456305225984,
  0.6888392356683349,
  0.6888328419384366,
  0.6888264493323096,
  0.6888200578493595,
  0.6888136674889918,
  0.6888072782506132,
  0.6888008901336309,
  0.6887945031374517,
  0.6887881172614835,
  0.6887817325051336,
  0.6887753488678109,
  0.6887689663489238,
  0.6887625849478818,
  0.6887562046640934,
  0.6887498254969691,
  0.6887434474459192,
  0.6887370705103535,
  0.6887306946896832,
  0.6887243199833197,
  0.6887179463906749,
  0.6887115739111601,
  0.6887052025441883,
  0.688698832289172,
  0.6886924631455241,
  0.6886860951126584,
  0.6886797281899885,
  0.6886733623769289,
  0.6886669976728934,
  0.688660634077298,
  0.6886542715895576,
  0.6886479102090873,
  0.6886415499353039,
  0.6886351907676235,
  0.6886288327054628,
  0.688622475748239,
  0.6886161198953701,
  0.6886097651462728,
  0.6886034115003662,
  0.6885970589570687,
  0.6885907075157992,
  0.6885843571759768,
  0.688578007937022,
  0.688571659798354,
  0.6885653127593935,
  0.6885589668195611,
  0.688552621978278,
  0.6885462782349664,
  0.6885399355890471,
  0.6885335940399426,
  0.688527253587076,
  0.6885209142298696,
  0.6885145759677471,
  0.6885082388001322,
  0.6885019027264487,
  0.688495567746121,
  0.6884892338585739,
  0.6884829010632325,
  0.6884765693595224,
  0.6884702387468692,
  0.6884639092246994,
  0.688457580792439,
  0.6884512534495155,
  0.688444927195356,
  0.6884386020293876,
  0.688432277951039,
  0.6884259549597384,
  0.6884196330549139,
  0.6884133122359953,
  0.6884069925024113,
  0.6884006738535923,
  0.6883943562889684,
  0.6883880398079698,
  0.6883817244100272,
  0.6883754100945724,
  0.6883690968610363,
  0.6883627847088507,
  0.6883564736374485,
  0.6883501636462622,
  0.6883438547347244,
  0.6883375469022686,
  0.6883312401483285,
  0.6883249344723381,
  0.6883186298737317,
  0.6883123263519442,
  0.6883060239064109,
  0.6882997225365669,
  0.688293422241848,
  0.6882871230216905,
  0.688280824875531,
  0.688274527802806,
  0.6882682318029532,
  0.6882619368754096,
  0.6882556430196136,
  0.6882493502350031,
  0.6882430585210174,
  0.6882367678770945,
  0.6882304783026743,
  0.6882241897971964,
  0.6882179023601008,
  0.688211615990828,
  0.6882053306888185,
  0.6881990464535135,
  0.6881927632843541,
  0.6881864811807826,
  0.688180200142241,
  0.6881739201681716,
  0.6881676412580173,
  0.688161363411221,
  0.6881550866272266,
  0.688148810905478,
  0.6881425362454192,
  0.6881362626464946,
  0.6881299901081493,
  0.6881237186298288,
  0.6881174482109782,
  0.688111178851044,
  0.6881049105494719,
  0.6880986433057091,
  0.6880923771192019,
  0.6880861119893985,
  0.6880798479157462,
  0.6880735848976928,
  0.6880673229346868,
  0.6880610620261772,
  0.6880548021716124,
  0.6880485433704425,
  0.6880422856221169,
  0.688036028926086,
  0.6880297732817994,
  0.688023518688709,
  0.6880172651462653,
  0.6880110126539196,
  0.6880047612111242,
  0.6879985108173311,
  0.6879922614719927,
  0.6879860131745617,
  0.6879797659244915,
  0.6879735197212354,
  0.6879672745642478,
  0.6879610304529822,
  0.6879547873868935,
  0.6879485453654369,
  0.6879423043880668,
  0.6879360644542397,
  0.6879298255634108,
  0.6879235877150365,
  0.687917350908574,
  0.6879111151434792,
  0.68790488041921,
  0.6878986467352243,
  0.6878924140909791,
  0.6878861824859337,
  0.6878799519195458,
  0.6878737223912752,
  0.6878674939005806,
  0.687861266446922,
  0.687855040029759,
  0.6878488146485524,
  0.6878425903027628,
  0.6878363669918508,
  0.6878301447152777,
  0.6878239234725055,
  0.6878177032629961,
  0.6878114840862118,
  0.6878052659416156,
  0.6877990488286697,
  0.6877928327468382,
  0.6877866176955842,
  0.6877804036743725,
  0.6877741906826668,
  0.6877679787199319,
  0.6877617677856327,
  0.6877555578792349,
  0.6877493490002041,
  0.687743141148006,
  0.6877369343221074,
  0.6877307285219747,
  0.687724523747075,
  0.6877183199968756,
  0.6877121172708444,
  0.6877059155684492,
  0.6876997148891582,
  0.6876935152324404,
  0.6876873165977645,
  0.6876811189846005,
  0.6876749223924173,
  0.6876687268206851,
  0.6876625322688749,
  0.6876563387364565,
  0.6876501462229011,
  0.6876439547276804,
  0.6876377642502658,
  0.6876315747901297,
  0.6876253863467437,
  0.6876191989195807,
  0.6876130125081141,
  0.6876068271118165,
  0.6876006427301624,
  0.6875944593626252,
  0.6875882770086794,
  0.6875820956677994,
  0.6875759153394605,
  0.6875697360231376,
  0.6875635577183067,
  0.6875573804244435,
  0.6875512041410243,
  0.6875450288675257,
  0.6875388546034246,
  0.6875326813481982,
  0.6875265091013244,
  0.6875203378622806,
  0.6875141676305453,
  0.6875079984055974,
  0.6875018301869149,
  0.6874956629739777,
  0.6874894967662654,
  0.687483331563257,
  0.6874771673644338,
  0.6874710041692756,
  0.6874648419772635,
  0.6874586807878786,
  0.6874525206006021,
  0.6874463614149161,
  0.6874402032303029,
  0.6874340460462444,
  0.6874278898622237,
  0.6874217346777238,
  0.6874155804922284,
  0.6874094273052211,
  0.6874032751161857,
  0.687397123924607,
  0.6873909737299688,
  0.6873848245317573,
  0.6873786763294574,
  0.6873725291225545,
  0.6873663829105352,
  0.6873602376928849,
  0.687354093469091,
  0.6873479502386403,
  0.6873418080010198,
  0.6873356667557176,
  0.6873295265022208,
  0.6873233872400182,
  0.6873172489685984,
  0.6873111116874501,
  0.6873049753960627,
  0.6872988400939255,
  0.6872927057805283,
  0.6872865724553615,
  0.6872804401179151,
  0.68727430876768,
  0.6872681784041478,
  0.6872620490268097,
  0.687255920635157,
  0.6872497932286825,
  0.6872436668068779,
  0.6872375413692363,
  0.68723141691525,
  0.6872252934444134,
  0.6872191709562195,
  0.6872130494501623,
  0.6872069289257362,
  0.6872008093824358,
  0.6871946908197555,
  0.6871885732371911,
  0.6871824566342382,
  0.6871763410103922,
  0.6871702263651496,
  0.6871641126980065,
  0.6871580000084602,
  0.6871518882960073,
  0.6871457775601456,
  0.6871396678003725,
  0.6871335590161864,
  0.6871274512070853,
  0.6871213443725684,
  0.6871152385121341,
  0.6871091336252817,
  0.6871030297115114,
  0.6870969267703225,
  0.6870908248012159,
  0.6870847238036913,
  0.68707862377725,
  0.6870725247213932,
  0.6870664266356225,
  0.6870603295194394,
  0.6870542333723462,
  0.6870481381938451,
  0.6870420439834392,
  0.6870359507406308,
  0.687029858464924,
  0.6870237671558223,
  0.6870176768128294,
  0.6870115874354497,
  0.6870054990231881,
  0.6869994115755486,
  0.6869933250920375,
  0.6869872395721597,
  0.6869811550154208,
  0.6869750714213273,
  0.6869689887893855,
  0.6869629071191022,
  0.6869568264099849,
  0.6869507466615399,
  0.6869446678732757,
  0.6869385900446999,
  0.6869325131753212,
  0.6869264372646477,
  0.6869203623121884,
  0.6869142883174523,
  0.6869082152799498,
  0.6869021431991895,
  0.6868960720746824,
  0.6868900019059384,
  0.6868839326924686,
  0.6868778644337838,
  0.6868717971293953,
  0.686865730778815,
  0.6868596653815545,
  0.6868536009371263,
  0.6868475374450429,
  0.6868414749048171,
  0.6868354133159618,
  0.6868293526779911,
  0.6868232929904182,
  0.6868172342527574,
  0.6868111764645234,
  0.68680511962523,
  0.6867990637343928,
  0.6867930087915272,
  0.6867869547961485,
  0.6867809017477724,
  0.6867748496459155,
  0.686768798490094,
  0.6867627482798252,
  0.6867566990146253,
  0.6867506506940125,
  0.6867446033175041,
  0.686738556884618,
  0.6867325113948729,
  0.686726466847787,
  0.6867204232428794,
  0.6867143805796693,
  0.6867083388576762,
  0.6867022980764199,
  0.6866962582354202,
  0.6866902193341977,
  0.6866841813722733,
  0.6866781443491677,
  0.6866721082644022,
  0.6866660731174986,
  0.6866600389079782,
  0.686654005635364,
  0.6866479732991783,
  0.6866419418989436,
  0.6866359114341828,
  0.6866298819044195,
  0.6866238533091775,
  0.686617825647981,
  0.6866117989203535,
  0.6866057731258198,
  0.6865997482639055,
  0.686593724334135,
  0.6865877013360339,
  0.686581679269128,
  0.6865756581329433,
  0.686569637927006,
  ...])

w.shape

(14, 1)

cost_lst[iter_num-1]

0.365635134439536

#绘制误差曲线
plt.plot(range(iter_num),cost_lst,"b-o")

[<matplotlib.lines.Line2D at 0x1422d44cdc0>]

【Python机器学习】实验03 逻辑回归,《机器学习》,机器学习,python,回归

这时要重新绘图了
items

X2

array([[ 1.00000000e+00,  6.99560000e-01,  4.89384194e-01, ...,
         1.28625106e-03,  1.34745327e-04,  9.42624411e-05],
       [ 1.00000000e+00,  6.84940000e-01,  4.69142804e-01, ...,
         4.03513411e-03, -7.97681228e-04, -5.46363780e-04],
       [ 1.00000000e+00,  6.92250000e-01,  4.79210063e-01, ...,
         2.18864648e-02, -9.76055545e-03, -6.75674451e-03],
       ...,
       [ 1.00000000e+00,  9.99270000e-01,  9.98540533e-01, ...,
         2.34349278e-01, -1.13696444e-01, -1.13613445e-01],
       [ 1.00000000e+00,  9.99270000e-01,  9.98540533e-01, ...,
         4.00913674e-05, -2.54406238e-07, -2.54220521e-07],
       [ 1.00000000e+00, -3.06120000e-02,  9.37094544e-04, ...,
         3.75068364e-04,  2.53215646e-01, -7.75143736e-03]])

X2.shape,w.shape

((118, 14), (14, 1))

y_pred=[1 if item>=0.5 else 0  for item in sigmoid(X2@w).ravel()]
y_pred=np.array(y_pred)
np.sum(y_pred==y2)/y2.shape[0]

0.8305084745762712

2.6 定义正则化项的代价函数

regularized cost（正则化代价函数）

$J\left( w \right)=\frac{1}{m}\sum\limits_{i=1}^{m}{[-{{y}^{(i)}}\log \left( {{h}}\left( {{x}^{(i)}} \right) \right)-\left( 1-{{y}^{(i)}} \right)\log \left( 1-{{h}}\left( {{x}^{(i)}} \right) \right)]}+\frac{\lambda }{2m}\sum\limits_{j=1}^{n}{w _{j}^{2}}$

w[:,0]

array([ 3.03503577,  3.20158942, -4.0495866 , -1.04983379, -3.95636068,
        2.0490215 , -3.40302089, -0.79821365, -1.23393575, -7.32541507,
       -1.41115593, -1.80717912, -0.54355034,  0.11775491])

#代价函数构造
def cost_reg(X,w,y,lambd):
    #当X(m,n+1),y(m,),w(n+1,1)
    y_hat=sigmoid(X@w)
    right1=np.multiply(y.ravel(),np.log(y_hat).ravel())+np.multiply((1-y).ravel(),np.log(1-y_hat).ravel())
    right2=(lambd/(2*X.shape[0]))*np.sum(np.power(w[1:,0],2))
    cost=-np.sum(right1)/X.shape[0]+right2
    return cost

cost(X2,w,y2)

0.365635134439536

lambd=2
cost_reg(X2,w,y2,lambd)

1.3874260376493517

2.7 定义正则化的梯度下降算法

如果我们要使用梯度下降法令这个代价函数最小化，因为我们未对 ${{w }_{0}}$ 进行正则化，所以梯度下降算法将分两种情形：

$\begin{align} & 重复\text{ }直到\text{ }收敛\text{ }\!\!\{\!\!\text{ } \\ & \text{ }{{w }_{0}}:={{w }_{0}}-a\frac{1}{m}\sum\limits_{i=1}^{m}{[{{h}}\left( {{x}^{(i)}} \right)-{{y}^{(i)}}]x_{_{0}}^{(i)}} \\ & \text{ }{{w }_{j}}:={{w }_{j}}-a[\frac{1}{m}\sum\limits_{i=1}^{m}{[{{h}}\left( {{x}^{(i)}} \right)-{{y}^{(i)}}]x_{j}^{(i)}}+\frac{\lambda }{m}{{w }_{j}}] \\ & \text{ }\!\!\}\!\!\text{ } \\ & 重复 \\ \end{align}$

对上面的算法中 j=1,2,…,n 时的更新式子进行调整可得：
$}_{j}}:={{w }_{j}}(1-a\frac{\lambda }{m})-a\frac{1}{m}\sum\limits_{i=1}^{m}{({{h}_{w }}\left( {{x}^{(i)}} \right)-{{y}^{(i)}})x_{j}^{(i)}}$

def grandient_reg(X,w,y,iter_num,alpha,lambd):
    y=y.reshape((X.shape[0],1))
    w=np.zeros((X.shape[1],1))
    cost_lst=[] 
    for i in range(iter_num):
        y_pred=h(X,w)-y
        temp=np.zeros((X.shape[1],1))
        for j in range(0,X.shape[1]):
            if j==0:
                right_0=np.multiply(y_pred.ravel(),X[:,0])
                gradient_0=1/(X.shape[0])*(np.sum(right_0))
                temp[j,0]=w[j,0]-alpha*(gradient_0)
            else:
                right=np.multiply(y_pred.ravel(),X[:,j])
                reg=(lambd/X.shape[0])*w[j,0]
                gradient=1/(X.shape[0])*(np.sum(right))
                temp[j,0]=w[j,0]-alpha*(gradient+reg)          
        w=temp
        cost_lst.append(cost_reg(X,w,y,lambd))
    return w,cost_lst

iter_num,alpha,lambd=600000,0.001,1
w2,cost_lst=grandient_reg(X2,w,y2,iter_num,alpha,lambd)

plt.plot(range(iter_num),cost_lst)

[<matplotlib.lines.Line2D at 0x1422dddef40>]

【Python机器学习】实验03 逻辑回归,《机器学习》,机器学习,python,回归

请注意等式中的"reg" 项。还注意到另外的一个“学习率”参数。这是一种超参数，用来控制正则化项。现在我们需要添加正则化梯度函数：

就像在第一部分中做的一样，初始化变量。

实验1 计算基于正则化得到的准确率

y_pred=[1 if item>=0.5 else 0  for item in sigmoid(X2@w).ravel()]
y_pred=np.array(y_pred)
np.sum(y_pred==y2)/y2.shape[0]

0.8305084745762712

现在，让我们尝试调用新的默认为0的 $w$ 的正则化函数，以确保计算工作正常。最后，我们可以使用第1部分中的预测函数来查看我们的方案在训练数据上的准确度。

2.8 试试sklearn

from sklearn import linear_model#调用sklearn的线性回归包
model = linear_model.LogisticRegression(penalty='l2', C=1.0)
model.fit(X2, y2.ravel())

LogisticRegression()

model.score(X2, y2)

0.8389830508474576

参考

[1] Andrew Ng. Machine Learning[EB/OL]. StanfordUniversity,2014.https://www.coursera.org/course/ml

[2] 李航. 统计学习方法[M]. 北京: 清华大学出版社,2019.

import sklearn.datasets as datasets
from sklearn.linear_model import LogisticRegression
import matplotlib.pyplot as plt

3.1 准备数据

X, y = datasets.make_blobs(n_samples=200, n_features=2, centers=2, random_state=0)

X.shape, y.shape

((200, 2), (200,))

array([[ 2.8219307 ,  1.25395648],
       [ 1.65581849,  1.26771955],
       [ 3.12377692,  0.44427786],
       [ 1.4178305 ,  0.50039185],
       [ 2.50904929,  5.7731461 ],
       [ 0.30380963,  3.94423417],
       [ 1.12031365,  5.75806083],
       [ 0.08848433,  2.32299086],
       [ 1.92238694,  0.59987278],
       [-0.65392827,  4.76656958],
       [ 1.45895348,  0.84509636],
       [ 0.51447051,  0.96092565],
       [ 1.35269561,  3.20438654],
       [-0.27652528,  5.08127768],
       [ 2.15299249,  1.48061734],
       [ 0.17286041,  3.61423755],
       [-0.20029671, -0.12484318],
       [ 3.52184624,  1.7502156 ],
       [ 2.5763324 ,  0.32187569],
       [ 2.89689879,  0.64820508],
       [ 1.36742991, -0.31641374],
       [-0.33963733,  3.84220272],
       [ 2.07592967,  4.95905106],
       [ 0.206354  ,  4.84303652],
       [ 2.89921211,  5.78430212],
       [ 0.340424  ,  4.98022062],
       [ 1.78753398, -0.23034767],
       [ 1.18454506,  5.28042636],
       [ 1.61434489,  0.61730816],
       [-0.60390472,  1.50398318],
       [-0.19685333,  6.24740851],
       [ 0.72100905, -0.44905385],
       [ 2.96544643,  1.21488188],
       [ 1.06975678, -0.57417135],
       [ 0.90802847,  6.01713005],
       [-0.17119857,  3.86596728],
       [ 1.36321767,  2.43404071],
       [ 1.24190326, -0.56876067],
       [ 1.33263648,  5.0103605 ],
       [ 0.62835793,  4.4601363 ],
       [ 0.70826671,  5.10624372],
       [ 2.8285205 , -0.28621698],
       [ 1.57561171,  1.51802196],
       [ 0.94808785,  4.7321192 ],
       [ 1.0427873 ,  4.60625923],
       [ 2.19722068,  0.57833524],
       [-0.29421492,  5.27318404],
       [ 0.02458305,  2.96215652],
       [ 2.16429987,  4.62072994],
       [ 4.31457647,  0.85540651],
       [ 0.86640826,  0.39084731],
       [ 1.5528609 ,  4.09548857],
       [ 1.44193252,  2.76754364],
       [ 0.93698726,  3.13569383],
       [ 2.21177406,  1.1298447 ],
       [ 0.46546494,  3.12315514],
       [ 3.13950603,  5.64031528],
       [ 0.9867701 ,  6.08965782],
       [ 1.74438135,  0.99506383],
       [ 0.89791226,  0.58537141],
       [ 2.74904067,  0.73809022],
       [ 4.01117983,  1.28775698],
       [-0.09448254,  5.35823905],
       [ 0.62227617,  2.92883603],
       [ 3.35941485,  5.24826681],
       [ 2.1047625 ,  1.39150044],
       [ 1.01001416,  2.10880895],
       [ 2.63378902,  1.24731812],
       [ 2.15504965,  4.12386249],
       [ 0.28170222,  4.15415279],
       [ 4.35918422, -0.16235216],
       [ 0.4666179 ,  3.86571303],
       [ 0.11898772,  1.08644226],
       [ 1.69057398,  1.05436752],
       [ 1.92156596,  1.97540747],
       [ 2.84159548,  0.43124456],
       [ 1.89760051,  3.15438716],
       [ 0.74874067,  2.55579434],
       [ 0.1631238 ,  2.57750473],
       [ 1.45661358, -0.21823333],
       [ 1.14294357,  4.93881876],
       [ 2.03824711,  1.2768154 ],
       [-1.57671974,  4.95740592],
       [-0.73000011,  6.25456272],
       [ 1.37125662,  2.55721446],
       [ 2.84382904,  5.20983199],
       [-0.51498751,  4.74317903],
       [ 2.01309607,  0.61077647],
       [ 1.67038771,  0.99201525],
       [ 1.59167155,  1.37914513],
       [ 1.37861172,  3.61897724],
       [-0.02394527,  2.75901623],
       [ 0.11504439,  6.21385228],
       [ 2.11567076,  3.06896151],
       [ 1.91931782,  2.03455502],
       [ 2.03958541,  1.05859183],
       [ 1.84836385,  1.77784257],
       [ 0.52073758,  4.32126649],
       [ 1.0220286 ,  4.11660348],
       [ 1.2911236 , -0.54012781],
       [ 0.34194798,  3.94104616],
       [ 2.5490093 ,  0.78155972],
       [ 1.15369622,  3.90200639],
       [ 0.60708824,  4.06440815],
       [-0.63762777,  4.09104705],
       [ 1.28933778,  3.44969159],
       [-0.12811326,  4.35595241],
       [ 0.08080352,  4.69068983],
       [ 3.20759909,  1.97728225],
       [ 0.06344785,  5.42080362],
       [ 2.80245586, -0.2912813 ],
       [ 2.20656076,  5.50616718],
       [ 1.7373078 ,  4.42546234],
       [ 1.70536064,  4.43277024],
       [ 0.47823763,  6.23331938],
       [ 2.6225578 ,  0.67498856],
       [ 0.21219797,  0.41968966],
       [ 1.76343016,  0.13617145],
       [ 1.09932252,  0.55168188],
       [ 1.86461403,  0.50281415],
       [ 1.59034945,  5.225994  ],
       [ 2.48152625,  1.57457169],
       [ 0.58894326,  4.00148458],
       [ 1.35056725,  1.84092438],
       [ 0.3571617 ,  1.28494414],
       [ 2.7216506 ,  0.43694387],
       [ 1.92352205,  4.14877723],
       [ 2.0309414 ,  0.15963275],
       [ 2.69858199, -0.67295975],
       [ 1.83310069,  3.65276173],
       [ 1.45795145,  0.65974193],
       [ 1.37227679,  3.21072582],
       [ 0.54111653,  6.15305106],
       [ 2.57915855,  0.98608575],
       [ 0.23151526,  3.47734879],
       [ 2.84382807,  3.32650945],
       [-0.24916544,  5.1481503 ],
       [ 1.40285894,  0.50671028],
       [ 2.74508569,  2.19950989],
       [ 3.70340245,  1.06189142],
       [ 1.42013331,  4.63746165],
       [ 0.47232912,  1.50804304],
       [ 1.8971289 ,  4.62251498],
       [ 0.10547293,  3.72493766],
       [ 2.32978388,  0.00674858],
       [ 1.60150153,  2.70172967],
       [ 0.30193742,  4.33561789],
       [-0.31658683,  4.5708382 ],
       [ 2.34161121,  1.50650749],
       [ 1.94472686,  1.91783637],
       [ 1.40297392,  0.37647435],
       [ 0.06897171,  4.35573272],
       [ 1.74806063,  5.12729148],
       [ 1.49954674,  4.132241  ],
       [ 0.63120661,  0.40434378],
       [ 1.27450825,  5.63017322],
       [ 0.66471755,  4.35995267],
       [ 1.42717996,  0.41663654],
       [ 2.9871159 ,  1.23762864],
       [ 1.33566313,  0.08467067],
       [ 0.92844171,  0.16698591],
       [ 2.46452227,  6.1996765 ],
       [ 2.85942078,  2.95602827],
       [ 2.69539905, -0.71929238],
       [ 1.70183577, -0.71881053],
       [ 1.11082127,  0.48761397],
       [ 0.23670708,  5.84680192],
       [ 1.1312175 ,  4.68194985],
       [ 0.33265168,  2.08038418],
       [-0.07228289,  2.88376939],
       [ 1.74625455, -0.77834015],
       [ 1.93710348,  0.21748546],
       [ 3.41979937,  0.20821448],
       [ 1.10318217,  4.70577669],
       [ 2.33570923, -0.09545995],
       [ 1.64856484,  4.71124916],
       [ 1.92569089,  4.39133857],
       [ 0.57309313,  5.5262324 ],
       [ 3.54975207, -1.17232137],
       [ 2.45431387, -1.8749291 ],
       [ 0.89908509,  1.67886176],
       [ 1.84070628,  3.56162231],
       [ 1.99364112,  0.79035838],
       [ 2.102906  ,  3.22385582],
       [ 0.87305123,  4.71438583],
       [ 0.5626511 ,  3.55633252],
       [ 2.75372467,  0.90143455],
       [ 2.09389807, -0.75905144],
       [ 1.32967014, -0.4857003 ],
       [-0.05797276,  4.98538185],
       [ 1.51240605,  1.31371371],
       [ 0.87781755,  3.64030904],
       [ 0.29937694,  1.34859812],
       [ 2.33519212,  0.79951327],
       [ 2.91319145,  2.03876553],
       [ 2.74680627,  1.5924128 ],
       [ 2.47034915,  4.09862906],
       [ 3.2460247 ,  2.84942165],
       [ 1.9263585 ,  4.15243012],
       [-0.18887976,  5.20461381]])

plt.scatter(X[:, 0], X[:, 1], c=y)

<matplotlib.collections.PathCollection at 0x142327368e0>

【Python机器学习】实验03 逻辑回归,《机器学习》,机器学习,python,回归

实验2 完成3.2 调用逻辑回归模型完成分类

3.2 调用普通的逻辑回归模型来进行多分类(调用1.4的梯度下降算法)

X=np.insert(X,0,1,axis=1)
X

array([[ 1.        ,  2.8219307 ,  1.25395648],
       [ 1.        ,  1.65581849,  1.26771955],
       [ 1.        ,  3.12377692,  0.44427786],
       [ 1.        ,  1.4178305 ,  0.50039185],
       [ 1.        ,  2.50904929,  5.7731461 ],
       [ 1.        ,  0.30380963,  3.94423417],
       [ 1.        ,  1.12031365,  5.75806083],
       [ 1.        ,  0.08848433,  2.32299086],
       [ 1.        ,  1.92238694,  0.59987278],
       [ 1.        , -0.65392827,  4.76656958],
       [ 1.        ,  1.45895348,  0.84509636],
       [ 1.        ,  0.51447051,  0.96092565],
       [ 1.        ,  1.35269561,  3.20438654],
       [ 1.        , -0.27652528,  5.08127768],
       [ 1.        ,  2.15299249,  1.48061734],
       [ 1.        ,  0.17286041,  3.61423755],
       [ 1.        , -0.20029671, -0.12484318],
       [ 1.        ,  3.52184624,  1.7502156 ],
       [ 1.        ,  2.5763324 ,  0.32187569],
       [ 1.        ,  2.89689879,  0.64820508],
       [ 1.        ,  1.36742991, -0.31641374],
       [ 1.        , -0.33963733,  3.84220272],
       [ 1.        ,  2.07592967,  4.95905106],
       [ 1.        ,  0.206354  ,  4.84303652],
       [ 1.        ,  2.89921211,  5.78430212],
       [ 1.        ,  0.340424  ,  4.98022062],
       [ 1.        ,  1.78753398, -0.23034767],
       [ 1.        ,  1.18454506,  5.28042636],
       [ 1.        ,  1.61434489,  0.61730816],
       [ 1.        , -0.60390472,  1.50398318],
       [ 1.        , -0.19685333,  6.24740851],
       [ 1.        ,  0.72100905, -0.44905385],
       [ 1.        ,  2.96544643,  1.21488188],
       [ 1.        ,  1.06975678, -0.57417135],
       [ 1.        ,  0.90802847,  6.01713005],
       [ 1.        , -0.17119857,  3.86596728],
       [ 1.        ,  1.36321767,  2.43404071],
       [ 1.        ,  1.24190326, -0.56876067],
       [ 1.        ,  1.33263648,  5.0103605 ],
       [ 1.        ,  0.62835793,  4.4601363 ],
       [ 1.        ,  0.70826671,  5.10624372],
       [ 1.        ,  2.8285205 , -0.28621698],
       [ 1.        ,  1.57561171,  1.51802196],
       [ 1.        ,  0.94808785,  4.7321192 ],
       [ 1.        ,  1.0427873 ,  4.60625923],
       [ 1.        ,  2.19722068,  0.57833524],
       [ 1.        , -0.29421492,  5.27318404],
       [ 1.        ,  0.02458305,  2.96215652],
       [ 1.        ,  2.16429987,  4.62072994],
       [ 1.        ,  4.31457647,  0.85540651],
       [ 1.        ,  0.86640826,  0.39084731],
       [ 1.        ,  1.5528609 ,  4.09548857],
       [ 1.        ,  1.44193252,  2.76754364],
       [ 1.        ,  0.93698726,  3.13569383],
       [ 1.        ,  2.21177406,  1.1298447 ],
       [ 1.        ,  0.46546494,  3.12315514],
       [ 1.        ,  3.13950603,  5.64031528],
       [ 1.        ,  0.9867701 ,  6.08965782],
       [ 1.        ,  1.74438135,  0.99506383],
       [ 1.        ,  0.89791226,  0.58537141],
       [ 1.        ,  2.74904067,  0.73809022],
       [ 1.        ,  4.01117983,  1.28775698],
       [ 1.        , -0.09448254,  5.35823905],
       [ 1.        ,  0.62227617,  2.92883603],
       [ 1.        ,  3.35941485,  5.24826681],
       [ 1.        ,  2.1047625 ,  1.39150044],
       [ 1.        ,  1.01001416,  2.10880895],
       [ 1.        ,  2.63378902,  1.24731812],
       [ 1.        ,  2.15504965,  4.12386249],
       [ 1.        ,  0.28170222,  4.15415279],
       [ 1.        ,  4.35918422, -0.16235216],
       [ 1.        ,  0.4666179 ,  3.86571303],
       [ 1.        ,  0.11898772,  1.08644226],
       [ 1.        ,  1.69057398,  1.05436752],
       [ 1.        ,  1.92156596,  1.97540747],
       [ 1.        ,  2.84159548,  0.43124456],
       [ 1.        ,  1.89760051,  3.15438716],
       [ 1.        ,  0.74874067,  2.55579434],
       [ 1.        ,  0.1631238 ,  2.57750473],
       [ 1.        ,  1.45661358, -0.21823333],
       [ 1.        ,  1.14294357,  4.93881876],
       [ 1.        ,  2.03824711,  1.2768154 ],
       [ 1.        , -1.57671974,  4.95740592],
       [ 1.        , -0.73000011,  6.25456272],
       [ 1.        ,  1.37125662,  2.55721446],
       [ 1.        ,  2.84382904,  5.20983199],
       [ 1.        , -0.51498751,  4.74317903],
       [ 1.        ,  2.01309607,  0.61077647],
       [ 1.        ,  1.67038771,  0.99201525],
       [ 1.        ,  1.59167155,  1.37914513],
       [ 1.        ,  1.37861172,  3.61897724],
       [ 1.        , -0.02394527,  2.75901623],
       [ 1.        ,  0.11504439,  6.21385228],
       [ 1.        ,  2.11567076,  3.06896151],
       [ 1.        ,  1.91931782,  2.03455502],
       [ 1.        ,  2.03958541,  1.05859183],
       [ 1.        ,  1.84836385,  1.77784257],
       [ 1.        ,  0.52073758,  4.32126649],
       [ 1.        ,  1.0220286 ,  4.11660348],
       [ 1.        ,  1.2911236 , -0.54012781],
       [ 1.        ,  0.34194798,  3.94104616],
       [ 1.        ,  2.5490093 ,  0.78155972],
       [ 1.        ,  1.15369622,  3.90200639],
       [ 1.        ,  0.60708824,  4.06440815],
       [ 1.        , -0.63762777,  4.09104705],
       [ 1.        ,  1.28933778,  3.44969159],
       [ 1.        , -0.12811326,  4.35595241],
       [ 1.        ,  0.08080352,  4.69068983],
       [ 1.        ,  3.20759909,  1.97728225],
       [ 1.        ,  0.06344785,  5.42080362],
       [ 1.        ,  2.80245586, -0.2912813 ],
       [ 1.        ,  2.20656076,  5.50616718],
       [ 1.        ,  1.7373078 ,  4.42546234],
       [ 1.        ,  1.70536064,  4.43277024],
       [ 1.        ,  0.47823763,  6.23331938],
       [ 1.        ,  2.6225578 ,  0.67498856],
       [ 1.        ,  0.21219797,  0.41968966],
       [ 1.        ,  1.76343016,  0.13617145],
       [ 1.        ,  1.09932252,  0.55168188],
       [ 1.        ,  1.86461403,  0.50281415],
       [ 1.        ,  1.59034945,  5.225994  ],
       [ 1.        ,  2.48152625,  1.57457169],
       [ 1.        ,  0.58894326,  4.00148458],
       [ 1.        ,  1.35056725,  1.84092438],
       [ 1.        ,  0.3571617 ,  1.28494414],
       [ 1.        ,  2.7216506 ,  0.43694387],
       [ 1.        ,  1.92352205,  4.14877723],
       [ 1.        ,  2.0309414 ,  0.15963275],
       [ 1.        ,  2.69858199, -0.67295975],
       [ 1.        ,  1.83310069,  3.65276173],
       [ 1.        ,  1.45795145,  0.65974193],
       [ 1.        ,  1.37227679,  3.21072582],
       [ 1.        ,  0.54111653,  6.15305106],
       [ 1.        ,  2.57915855,  0.98608575],
       [ 1.        ,  0.23151526,  3.47734879],
       [ 1.        ,  2.84382807,  3.32650945],
       [ 1.        , -0.24916544,  5.1481503 ],
       [ 1.        ,  1.40285894,  0.50671028],
       [ 1.        ,  2.74508569,  2.19950989],
       [ 1.        ,  3.70340245,  1.06189142],
       [ 1.        ,  1.42013331,  4.63746165],
       [ 1.        ,  0.47232912,  1.50804304],
       [ 1.        ,  1.8971289 ,  4.62251498],
       [ 1.        ,  0.10547293,  3.72493766],
       [ 1.        ,  2.32978388,  0.00674858],
       [ 1.        ,  1.60150153,  2.70172967],
       [ 1.        ,  0.30193742,  4.33561789],
       [ 1.        , -0.31658683,  4.5708382 ],
       [ 1.        ,  2.34161121,  1.50650749],
       [ 1.        ,  1.94472686,  1.91783637],
       [ 1.        ,  1.40297392,  0.37647435],
       [ 1.        ,  0.06897171,  4.35573272],
       [ 1.        ,  1.74806063,  5.12729148],
       [ 1.        ,  1.49954674,  4.132241  ],
       [ 1.        ,  0.63120661,  0.40434378],
       [ 1.        ,  1.27450825,  5.63017322],
       [ 1.        ,  0.66471755,  4.35995267],
       [ 1.        ,  1.42717996,  0.41663654],
       [ 1.        ,  2.9871159 ,  1.23762864],
       [ 1.        ,  1.33566313,  0.08467067],
       [ 1.        ,  0.92844171,  0.16698591],
       [ 1.        ,  2.46452227,  6.1996765 ],
       [ 1.        ,  2.85942078,  2.95602827],
       [ 1.        ,  2.69539905, -0.71929238],
       [ 1.        ,  1.70183577, -0.71881053],
       [ 1.        ,  1.11082127,  0.48761397],
       [ 1.        ,  0.23670708,  5.84680192],
       [ 1.        ,  1.1312175 ,  4.68194985],
       [ 1.        ,  0.33265168,  2.08038418],
       [ 1.        , -0.07228289,  2.88376939],
       [ 1.        ,  1.74625455, -0.77834015],
       [ 1.        ,  1.93710348,  0.21748546],
       [ 1.        ,  3.41979937,  0.20821448],
       [ 1.        ,  1.10318217,  4.70577669],
       [ 1.        ,  2.33570923, -0.09545995],
       [ 1.        ,  1.64856484,  4.71124916],
       [ 1.        ,  1.92569089,  4.39133857],
       [ 1.        ,  0.57309313,  5.5262324 ],
       [ 1.        ,  3.54975207, -1.17232137],
       [ 1.        ,  2.45431387, -1.8749291 ],
       [ 1.        ,  0.89908509,  1.67886176],
       [ 1.        ,  1.84070628,  3.56162231],
       [ 1.        ,  1.99364112,  0.79035838],
       [ 1.        ,  2.102906  ,  3.22385582],
       [ 1.        ,  0.87305123,  4.71438583],
       [ 1.        ,  0.5626511 ,  3.55633252],
       [ 1.        ,  2.75372467,  0.90143455],
       [ 1.        ,  2.09389807, -0.75905144],
       [ 1.        ,  1.32967014, -0.4857003 ],
       [ 1.        , -0.05797276,  4.98538185],
       [ 1.        ,  1.51240605,  1.31371371],
       [ 1.        ,  0.87781755,  3.64030904],
       [ 1.        ,  0.29937694,  1.34859812],
       [ 1.        ,  2.33519212,  0.79951327],
       [ 1.        ,  2.91319145,  2.03876553],
       [ 1.        ,  2.74680627,  1.5924128 ],
       [ 1.        ,  2.47034915,  4.09862906],
       [ 1.        ,  3.2460247 ,  2.84942165],
       [ 1.        ,  1.9263585 ,  4.15243012],
       [ 1.        , -0.18887976,  5.20461381]])

#调用梯度下降算法
iter_num,alpha=600000,0.001
w,cost_lst=grandient(X,y,iter_num,alpha)

#绘制误差曲线
plt.plot(range(iter_num),cost_lst,"b-o")

[<matplotlib.lines.Line2D at 0x1423849dc70>]

【Python机器学习】实验03 逻辑回归,《机器学习》,机器学习,python,回归

X[y==0,1]

array([ 2.50904929,  0.30380963,  1.12031365,  0.08848433, -0.65392827,
        1.35269561, -0.27652528,  0.17286041, -0.33963733,  2.07592967,
        0.206354  ,  2.89921211,  0.340424  ,  1.18454506, -0.19685333,
        0.90802847, -0.17119857,  1.33263648,  0.62835793,  0.70826671,
        0.94808785,  1.0427873 , -0.29421492,  2.16429987,  1.5528609 ,
        1.44193252,  0.93698726,  0.46546494,  3.13950603,  0.9867701 ,
       -0.09448254,  0.62227617,  3.35941485,  2.15504965,  0.28170222,
        0.4666179 ,  0.1631238 ,  1.14294357, -1.57671974, -0.73000011,
        2.84382904, -0.51498751,  1.37861172, -0.02394527,  0.11504439,
        2.11567076,  0.52073758,  1.0220286 ,  0.34194798,  1.15369622,
        0.60708824, -0.63762777,  1.28933778, -0.12811326,  0.08080352,
        0.06344785,  2.20656076,  1.7373078 ,  1.70536064,  0.47823763,
        1.59034945,  0.58894326,  1.92352205,  1.83310069,  1.37227679,
        0.54111653,  0.23151526,  2.84382807, -0.24916544,  1.42013331,
        1.8971289 ,  0.10547293,  1.60150153,  0.30193742, -0.31658683,
        0.06897171,  1.74806063,  1.49954674,  1.27450825,  0.66471755,
        2.46452227,  2.85942078,  0.23670708,  1.1312175 ,  0.33265168,
       -0.07228289,  1.10318217,  1.64856484,  1.92569089,  0.57309313,
        1.84070628,  2.102906  ,  0.87305123,  0.5626511 , -0.05797276,
        0.87781755,  2.47034915,  3.2460247 ,  1.9263585 , -0.18887976])

#绘制线性的决策边界
x_exmal=np.linspace(np.min(X[:,1]),np.max(X[:,1]),50)
x2=(-w[0,0]-w[1,0]*x_exmal)/(w[2,0])
plt.plot(x_exmal,x2,"r-o")
plt.scatter(X[y==1,1],X[y==1,2],color="b",marker="o")
plt.scatter(X[y==0,1],X[y==0,2],color="c",marker="^")
plt.show()

【Python机器学习】实验03 逻辑回归,《机器学习》,机器学习,python,回归

#计算准确率
y_pred=[1 if item>=0.5 else 0  for item in sigmoid(X@w).ravel()]
y_pred=np.array(y_pred)
np.sum(y_pred==y)/y.shape[0]

0.97

实验3 完成3.3 调用正则化的逻辑回归模型完成分类

3.3调用正则化的逻辑回归模型来进行多分类(调用2.7的梯度下降算法)

y.shape,X.shape,w.shape

((200,), (200, 3), (3, 1))

#调用梯度下降算法
iter_num,alpha,lambd=600000,0.001,1
w,cost_lst=grandient_reg(X,w,y,iter_num,alpha,lambd)

#绘制误差曲线
plt.plot(range(iter_num),cost_lst,"b-o")

[<matplotlib.lines.Line2D at 0x1423279f070>]

【Python机器学习】实验03 逻辑回归,《机器学习》,机器学习,python,回归

#绘制线性的决策边界
x_exmal=np.linspace(np.min(X[:,1]),np.max(X[:,1]),50)
x2=(-w[0,0]-w[1,0]*x_exmal)/(w[2,0])
plt.plot(x_exmal,x2,"r-o")
plt.scatter(X[y==1,1],X[y==1,2],color="b",marker="o")
plt.scatter(X[y==0,1],X[y==0,2],color="c",marker="^")
plt.show()

【Python机器学习】实验03 逻辑回归,《机器学习》,机器学习,python,回归文章来源地址https://www.toymoban.com/news/detail-671189.html

y.shape,X.shape,w.shape

((200,), (200, 3), (3, 1))

#计算准确率
y_pred=[1 if item>=0.5 else 0  for item in sigmoid(X@w).ravel()]
y_pred=np.array(y_pred)
np.sum(y_pred==y)/y.shape[0]

0.97

实验4 完成3.3 调用SKLEARN完成分类

3.4 调用SKLEARN

from sklearn.linear_model import LogisticRegression
clf = LogisticRegression().fit(X, y)
clf.score(X,y)

0.97

附：系列文章

实验	目录	直达链接
1	Numpy以及可视化回顾	https://want595.blog.csdn.net/article/details/131891689
2	线性回归	https://want595.blog.csdn.net/article/details/131892463
3	逻辑回归	https://want595.blog.csdn.net/article/details/131912053
4	多分类实践(基于逻辑回归)	https://want595.blog.csdn.net/article/details/131913690
5	机器学习应用实践-手动调参	https://want595.blog.csdn.net/article/details/131934812
6	贝叶斯推理	https://want595.blog.csdn.net/article/details/131947040
7	KNN最近邻算法	https://want595.blog.csdn.net/article/details/131947885
8	K-means无监督聚类	https://want595.blog.csdn.net/article/details/131952371
9	决策树	https://want595.blog.csdn.net/article/details/131991014
10	随机森林和集成学习	https://want595.blog.csdn.net/article/details/132003451
11	支持向量机	https://want595.blog.csdn.net/article/details/132010861
12	神经网络-感知器	https://want595.blog.csdn.net/article/details/132014769
13	基于神经网络的回归-分类实验	https://want595.blog.csdn.net/article/details/132127413
14	手写体卷积神经网络	https://want595.blog.csdn.net/article/details/132223494
15	将Lenet5应用于Cifar10数据集	https://want595.blog.csdn.net/article/details/132223751
16	卷积、下采样、经典卷积网络	https://want595.blog.csdn.net/article/details/132223985