Logistic回归_业界新闻

发布时间:2024-08-03 06:45

阅读量:0

Logistic回归模型：

适用于二分类或多分类问题，样本特征是数值型（否则需要转换为数值型）

策略：极大似然估计

算法：随机梯度或 BFGS算法（改进的拟牛顿法）

线性回归表达式：

$y_i = w\cdot x_i+b$

式子中 $x_i = (x_i^{(1)},x_i^{(2)},...,x_i^{(N)})$ ;w为N个特征权重组成的向量，即;b是第i个样本对应的偏置常数。

Sigmoid函数：

$g(z)=\frac{1}{1+e^{-z}}$

对数概率

$y=log(\frac{p}{1-p})$

$p = \frac{e^y}{1+e^y}$

$p=\frac{e^{wx+b}}{1+e^{wx+b}}$

Logistic 回归模型：

$p(y=1|x)=\frac{e^{wx+b}}{1+e^{wx+b}}$ ， $p(y=0|x)=\frac{1}{1+e^{wx+b}}$

构造似然函数：

$log(L)=\sum_{i=1}^{N}y_i(wx_i+b)+log(1-p_i)$

$log(L)=\sum_{i=1}^{N}y_i(wx_i+b)-log(1+e^{wx_i+b})$

$\hat{w},\hat{b}=argmax_{w,b}\sum_{i=1}^{N}y_i(wx_i+b)-log(1+e^{wx_i+b})$

Logistic回归优化：梯度下降，分别对权重w，偏置b求导数：

$\frac{\partial }{\partial w}lnL(w,b)=\frac{\partial }{\partial w}\sum_{i=1}^{N}y_i(wx_i+b)-ln(1+e^{wx_i+b})$

$\frac{\partial }{\partial b}lnL(w,b)=\frac{\partial }{\partial b}\sum_{i=1}^{N}y_i(wx_i+b)-ln(1+e^{wx_i+b})$

综上，可归纳Logistic回归的过程：

实例：鸢尾花数据集划分：

class Logistic_Regression:          def __init__(self):         self.coef_ = None         self.intercept_ = None         self._theta = None              def _sigmoid(self,t):         return 1./(1.+np.exp(-t))           def fit(self,X_train,y_train,eta = 0.01, n_iters =1e4):                  def J(theta,X_b,y):             y_hat = self._sigmoid(X_b.dot(theta))             try:                 return -np.sum(y*np.log(y_hat)  +(1-y)*np.log(1-y_hat)  )             except:                 return float('inf')                  def dJ(theta,X_b,y):             return X_b.T.dot(self._sigmoid(X_b.dot(theta))-y)              def gradient_descent(initia_theta,X_b,y, eta,n_iters =1e4,epsilon =1e-8 ):             theta = initia_theta             cur_iter = 0              while cur_iter < n_iters:                 gradient = dJ(theta,X_b, y)                 last_theta = theta                 theta = theta - eta * gradient                  if (abs(J(theta,X_b, y)-J(last_theta,X_b, y)) < epsilon):                     break                 cur_iter += 1              return theta          X_b = np.hstack([np.ones(len(X_train)).reshape(-1,1),X_train])         initia_theta = np.zeros(X_b.shape[1])         self._theta = gradient_descent(initia_theta,X_b,y_train,eta,n_iters)                  self.intercept_ = self._theta[0]         self.coef_ = self._theta[1:]                  return self          def predict_proba(self,X_predict):         X_b = np.hstack([np.ones(len(X_predict)).reshape(-1,1),X_predict])         return self._sigmoid(X_b.dot(self._theta))          def predict(self,X_predict):         proba = self.predict_proba(X_predict)         return np.array(proba >= 0.5,dtype = 'int')          def score(self,X_test,y_test):         y_predict = self.predict(X_test)         return accuracy_score(y_test, y_predict)          def __repr__(self):         return "LogisticRegression()"

可视化划分：

from sklearn import datasets iris = datasets.load_iris() X = iris.data y = iris.target X = X[y<2,:2] y = y[y<2] plot_decision_boundary(log_reg,X_test) plt.scatter(X_test[y_test==0,0],X_test[y_test==0,1]) plt.scatter(X_test[y_test==1,0],X_test[y_test==1,1]) plt.show()