回归预测

In [2]:
# 导入数据
from sklearn.datasets import load_boston
boston = load_boston()
print boston.DESCR

.. _boston_dataset:

Boston house prices dataset
---------------------------

**Data Set Characteristics:**  

    :Number of Instances: 506 

    :Number of Attributes: 13 numeric/categorical predictive. Median Value (attribute 14) is usually the target.

    :Attribute Information (in order):
        - CRIM     per capita crime rate by town
        - ZN       proportion of residential land zoned for lots over 25,000 sq.ft.
        - INDUS    proportion of non-retail business acres per town
        - CHAS     Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)
        - NOX      nitric oxides concentration (parts per 10 million)
        - RM       average number of rooms per dwelling
        - AGE      proportion of owner-occupied units built prior to 1940
        - DIS      weighted distances to five Boston employment centres
        - RAD      index of accessibility to radial highways
        - TAX      full-value property-tax rate per $10,000
        - PTRATIO  pupil-teacher ratio by town
        - B        1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town
        - LSTAT    % lower status of the population
        - MEDV     Median value of owner-occupied homes in $1000's

    :Missing Attribute Values: None

    :Creator: Harrison, D. and Rubinfeld, D.L.

This is a copy of UCI ML housing dataset.
https://archive.ics.uci.edu/ml/machine-learning-databases/housing/


This dataset was taken from the StatLib library which is maintained at Carnegie Mellon University.

The Boston house-price data of Harrison, D. and Rubinfeld, D.L. 'Hedonic
prices and the demand for clean air', J. Environ. Economics & Management,
vol.5, 81-102, 1978.   Used in Belsley, Kuh & Welsch, 'Regression diagnostics
...', Wiley, 1980.   N.B. Various transformations are used in the table on
pages 244-261 of the latter.

The Boston house-price data has been used in many machine learning papers that address regression
problems.   
     
.. topic:: References

   - Belsley, Kuh & Welsch, 'Regression diagnostics: Identifying Influential Data and Sources of Collinearity', Wiley, 1980. 244-261.
   - Quinlan,R. (1993). Combining Instance-Based and Model-Based Learning. In Proceedings on the Tenth International Conference of Machine Learning, 236-243, University of Massachusetts, Amherst. Morgan Kaufmann.
In [3]:
#分割数据
from sklearn.model_selection import train_test_split
import numpy as np

x = boston.data
y = boston.target

x_train, x_test, y_train, y_test = train_test_split(x,y,random_state=33,test_size=0.25)
print 'The max target value is', np.max(boston.target)
print 'The min target value is', np.min(boston.target)
print 'The average target value is', np.mean(boston.target)

The max target value is 50.0
The min target value is 5.0
The average target value is 22.5328063241
In [4]:
# max-min过大，标准化处理
from sklearn.preprocessing import StandardScaler
ss_x = StandardScaler()
ss_y = StandardScaler()

x_train = ss_x.fit_transform(x_train)
x_test = ss_x.transform(x_test)
y_train = ss_y.fit_transform(y_train[:,None])[:,0] #ss函数要求输入必须至少二维数组，此为等价格式
y_test = ss_y.transform(y_test[:,None])[:,0]

使用线性回归模型LinearRegression和SGDRegressor

In [5]:
from sklearn.linear_model import LinearRegression
lr = LinearRegression()
lr.fit(x_train,y_train)
lr_y_predict = lr.predict(x_test)

from sklearn.linear_model import SGDRegressor
sgdr = SGDRegressor()
sgdr.fit(x_train,y_train)
sgdr_y_predict = sgdr.predict(x_test)

C:\Users\spark\Anaconda2\lib\site-packages\sklearn\linear_model\stochastic_gradient.py:166: FutureWarning: max_iter and tol parameters have been added in SGDRegressor in 0.19. If both are left unset, they default to max_iter=5 and tol=None. If tol is not None, max_iter defaults to max_iter=1000. From 0.21, default max_iter will be 1000, and default tol will be 1e-3.
  FutureWarning)
In [6]:
# 性能评价：倾向于测量预测值与真实值的差距
# 准确率
print 'The value of default measurement of LinearRegression is', lr.score(x_test,y_test)
# R-squared，平均绝对误差MAE，均方误差MSE
# 评估LR
from sklearn.metrics import r2_score, mean_squared_error, mean_absolute_error
print 'The value of R-squared of LR is', r2_score(y_test,lr_y_predict)
print 'The MSE of LR is', mean_squared_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(lr_y_predict))
print 'The MAE of LR is', mean_absolute_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(lr_y_predict))

The value of default measurement of LinearRegression is 0.675795501453
The value of R-squared of LR is 0.675795501453
The MSE of LR is 25.1392365204
The MAE of LR is 3.53253254371
In [7]:
# 评估SGDR
print 'The value of default measurement of SGDR is', sgdr.score(x_test,y_test)
print 'The R-squared of SGDR is', r2_score(y_test,sgdr_y_predict)
print 'The MSE of SGDR is', mean_squared_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(sgdr_y_predict))
print 'The MAE of SGDR is', mean_absolute_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(sgdr_y_predict))

The value of default measurement of SGDR is 0.649378781018
The R-squared of SGDR is 0.649378781018
The MSE of SGDR is 27.187623221
The MAE of SGDR is 3.52558816548

使用支持向量机（回归）SVR

In [8]:
from sklearn.svm import SVR
# 使用线性核函数
linear_svr = SVR(kernel='linear')
linear_svr.fit(x_train,y_train)
linear_svr_y_predict = linear_svr.predict(x_test)
# 使用多项式核函数
poly_svr = SVR(kernel='poly')
poly_svr.fit(x_train,y_train)
poly_svr_y_predict = poly_svr.predict(x_test)
#使用径向基核函数
rbf_svr = SVR(kernel = 'rbf')
rbf_svr.fit(x_train,y_train)
rbf_svr_y_predict = rbf_svr.predict(x_test)
In [9]:
# 性能评价
from sklearn.metrics import r2_score, mean_absolute_error, mean_squared_error
print 'Linear-svr:', r2_score(y_test,linear_svr_y_predict), \
mean_squared_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(linear_svr_y_predict)), \
mean_absolute_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(linear_svr_y_predict))
print 'Poly-svr:', r2_score(y_test,poly_svr_y_predict), \
mean_squared_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(poly_svr_y_predict)), \
mean_absolute_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(poly_svr_y_predict))
print 'rbf-svr:', r2_score(y_test,rbf_svr_y_predict), \
mean_squared_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(rbf_svr_y_predict)), \
mean_absolute_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(rbf_svr_y_predict))

Linear-svr: 0.650659546422 27.0883110136 3.43280138776
Poly-svr: 0.403650651026 46.241700531 3.73840737105
rbf-svr: 0.755988741634 18.9209488615 2.60678199995

使用K近邻（回归）

In [10]:
from sklearn.neighbors import KNeighborsRegressor
uni_knr = KNeighborsRegressor(weights = 'uniform')
uni_knr.fit(x_train,y_train)
uni_knr_y_predict = uni_knr.predict(x_test)

dis_knr = KNeighborsRegressor(weights = 'distance')
dis_knr.fit(x_train,y_train)
dis_knr_y_predict = dis_knr.predict(x_test)
In [11]:
# 性能评价
from sklearn.metrics import r2_score, mean_absolute_error, mean_squared_error
print 'uni_knr:', r2_score(y_test,uni_knr_y_predict), \
mean_squared_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(uni_knr_y_predict)), \
mean_absolute_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(uni_knr_y_predict))
print 'dis_knr:', r2_score(y_test,dis_knr_y_predict), \
mean_squared_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(dis_knr_y_predict)), \
mean_absolute_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(dis_knr_y_predict))

uni_knr: 0.690721217635 23.9818771654 2.96503937008
dis_knr: 0.720109482142 21.7030730905 2.80112550221

回归树

In [15]:
from sklearn.tree import DecisionTreeRegressor
dtr = DecisionTreeRegressor()
dtr.fit(x_train,y_train)
dtr_y_predict = dtr.predict(x_test)
In [16]:
print 'dtr:', r2_score(y_test,dtr_y_predict), \
mean_squared_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(dtr_y_predict)), \
mean_absolute_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(dtr_y_predict))

dtr: 0.709782272857 22.5038582677 3.03070866142

集成模型

In [19]:
from sklearn.ensemble import RandomForestRegressor, ExtraTreesRegressor, GradientBoostingRegressor
# 随机森林
rfr = RandomForestRegressor()
rfr.fit(x_train,y_train)
rfr_y_predict = rfr.predict(x_test)
# 极端随机森林
etr = ExtraTreesRegressor()
etr.fit(x_train,y_train)
etr_y_predict = etr.predict(x_test)
# 梯度提升回归树
gbr = GradientBoostingRegressor()
gbr.fit(x_train,y_train)
gbr_y_predict = gbr.predict(x_test)



















In [20]:



    


print 'rfr:', r2_score(y_test,rfr_y_predict), \
mean_squared_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(rfr_y_predict)), \
mean_absolute_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(rfr_y_predict))



    















    






rfr: 0.822486731243 13.7646086614 2.37125984252



















In [22]:



    


print 'etr:', r2_score(y_test,etr_y_predict), \
mean_squared_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(etr_y_predict)), \
mean_absolute_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(etr_y_predict))
print np.sort(zip(etr.feature_importances_,boston.feature_names),axis=0)



    















    






etr: 0.766582112164 18.0995251969 2.63503937008
[['0.00448467872175' 'AGE']
 ['0.0122200231676' 'B']
 ['0.013315058436' 'CHAS']
 ['0.0144814833089' 'CRIM']
 ['0.0145930363402' 'DIS']
 ['0.0175846481952' 'INDUS']
 ['0.0253233081457' 'LSTAT']
 ['0.02959825521' 'NOX']
 ['0.0322054503129' 'PTRATIO']
 ['0.0452196083062' 'RAD']
 ['0.0512715618837' 'RM']
 ['0.314831141034' 'TAX']
 ['0.424871746937' 'ZN']]



















In [23]:



    


print 'gbr:', r2_score(y_test,gbr_y_predict), \
mean_squared_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(gbr_y_predict)), \
mean_absolute_error(ss_y.inverse_transform(y_test),ss_y.inverse_transform(gbr_y_predict))



    















    






gbr: 0.83902528947 12.4821874463 2.30248434325