一、提出问题¶
- 哪些单一特征群体的幸存率较大？
- 怎样的复合特征群体的幸存率较大？
- 最终选择的特征间如果存在较大相关性，是否会降低预测准确率？
二、数据加工¶
导入库¶
In [2]:
# 添加会用到的库
import numpy as np
import pandas as pd
import os
import random as rnd
print(os.listdir('C:/Users/spark/Desktop/ML101/project-titanic/raw_data'))
# 可视化
import seaborn as sns
import matplotlib.pyplot as plt
# magic命令，确保作图在当前页面显示，但同行代码不能有中文
%matplotlib inline
# 机器学习
from sklearn.linear_model import LogisticRegression # 分类：逻辑回归
from sklearn.linear_model import SGDClassifier # 分类：梯度下降
from sklearn.svm import LinearSVC # 分类：支持向量机
from sklearn.tree import DecisionTreeClassifier # 分类：决策树
from sklearn.ensemble import RandomForestClassifier # 分类：随机森林
from sklearn.ensemble import GradientBoostingClassifier # 分类：梯度提升决策树
from sklearn.neighbors import KNeighborsClassifier # 分类：k邻近
from sklearn.naive_bayes import GaussianNB
from sklearn.linear_model import Perceptron
查看字段¶
In [3]:
# 获取数据文件
train_df = pd.read_csv('C:/Users/spark/Desktop/ML101/project-titanic/raw_data/train.csv')
predict_df = pd.read_csv('C:/Users/spark/Desktop/ML101/project-titanic/raw_data/test.csv')
combined = [train_df, predict_df]
full_df = train_df.append(predict_df, ignore_index=True)
print(train_df.columns.values)
有一些二分类数据：survived，sex，embarked
还有一些数值型数据，可以分为离散的、连续的、时间序列的
- 离散的：sibsp(sibling/spouse)，parch（parents/children）
- 连续的：age，fare
In [4]:
train_df.head()
Out[4]:
survival:生存状态
pclass:船舱等级
embarked：登船地点
查看数据统计信息¶
In [5]:
full_df.info()
In [6]:
train_df.describe()
Out[6]:
三、数据探索¶
In [7]:
# 验证假设
# Sex
train_df[['Sex', 'Survived']].groupby(['Sex'], as_index=False)\
.mean().sort_values(by='Survived', ascending=False)
Out[7]:
In [8]:
# Age
g = sns.FacetGrid(train_df, col='Survived')
#g.map(plt.hist, 'Age', bins=20)
g.map(sns.distplot, 'Age',rug=True,
hist_kws={'color':'blue','label':'hist'},
kde_kws={'color':'red','label':'KDE'},)
g.add_legend()
Out[8]:
Observations¶
- 人数最多：15-35岁
- 幸存率高：婴儿（0-4岁），老年人（80岁）
- 幸存率低：15-25岁
In [9]:
# Pclass
train_df[['Pclass', 'Survived']].groupby(['Pclass'], as_index=False)\
.mean().sort_values(by='Survived', ascending=False)
Out[9]:
In [10]:
# Pclass&Age
g = sns.FacetGrid(train_df, col='Survived', row='Pclass', height=2.2, aspect=1.6)
g.map(plt.hist, 'Age', alpha=0.5, bins=20)
g.add_legend()
Out[10]:
In [11]:
# Embarked
g = sns.FacetGrid(train_df, row='Embarked', height=2.2, aspect=1.6)
g.map(sns.pointplot, 'Pclass', 'Survived', 'Sex', palette='deep')
g.add_legend()
Out[11]:
In [12]:
# Fare
g = sns.FacetGrid(train_df, row='Embarked', col='Survived', height=2.2, aspect=1.6)
g.map(sns.barplot, 'Sex', 'Fare', alpha=.5, ci=None)
g.add_legend()
Out[12]:
四、数据再加工¶
修复完整性和无效性¶
In [13]:
# 修复完整性和无效性
full_df['Age'].fillna(full_df['Age'].mean(), inplace=True)
full_df['Embarked'].fillna(full_df['Embarked'].dropna().mode()[0], inplace=True)
full_df['Fare'].fillna(full_df['Fare'].dropna().median(), inplace=True)
full_df.drop(['Cabin', 'Ticket'], axis=1, inplace=True)
full_df.info()
特征转换：将类别转换为数字¶
In [14]:
# 特征转换：将类别转换为数字
full_df['Sex'] = full_df['Sex'].map({'female':1, 'male':0}).astype(int)
full_df['Embarked'] = full_df['Embarked'].map({'S':0,'Q':1,'C':2}).astype(int)
In [15]:
full_df.head()
Out[15]:
相关系数矩阵¶
In [16]:
# 相关系数矩阵
r_df = full_df.corr()
r_df
Out[16]:
In [17]:
# 热力图
#import matplotlib.pyplot as plt
#import seaborn as sns
f,ax = plt.subplots(figsize=(6,3)) # 设定画幅
sns.heatmap(r_df, vmax =1, square=True, center=0) # 生成热力图
# 调整坐标标签方向
#label_x = ax.get_xticklabels()
#plt.setp(label_x , rotation = 90)
plt.show()
特征转换：划分数值区间（分箱）¶
In [18]:
# 划分数值区间
# Age
# 调试区间：查看年龄区间的幸存率
train_df['AgeBand'] = pd.cut(train_df['Age'], 4)
train_df[['AgeBand', 'Survived']].groupby(['AgeBand'], as_index=False)\
.mean().sort_values(by='AgeBand', ascending=True)
Out[18]:
In [19]:
# 按照上方区间，修改Age为分组编号
full_df['Age'] = full_df['Age'].map(lambda x:0 if x<=20 else x)
full_df['Age'] = full_df['Age'].map(lambda x:1 if 20<x<=40 else x)
full_df['Age'] = full_df['Age'].map(lambda x:2 if 40<x<=60 else x)
full_df['Age'] = full_df['Age'].map(lambda x:3 if 60<x else x)
full_df['Age'] = full_df['Age'].map(lambda x:int(x))
In [20]:
# Fare
# 调试区间：查看年龄区间的幸存率
train_df['FareBand'] = pd.qcut(train_df['Fare'], 4)
train_df[['FareBand', 'Survived']].groupby(['FareBand'], as_index=False)\
.mean().sort_values(by='FareBand', ascending=True)
Out[20]:
In [21]:
full_df['Fare'] = full_df['Fare'].map(lambda x:0 if x<=7.91 else x)
full_df['Fare'] = full_df['Fare'].map(lambda x:1 if 7.91<x<=14.454 else x)
full_df['Fare'] = full_df['Fare'].map(lambda x:2 if 14.454<x<=31.0 else x)
full_df['Fare'] = full_df['Fare'].map(lambda x:3 if 31.0<x else x)
full_df['Fare'] = full_df['Fare'].map(lambda x:int(x))
full_df.head()
Out[21]:
特征转换：从文本中截取关键字¶
In [22]:
# Name
# 从name中提取title
def title_split(name):
start = name.find(',') + 1
end = name.find('.')
return name[start:end].strip()
train_df['Title'] = train_df['Name'].map(title_split)
train_df['Title'].value_counts()
Out[22]:
In [23]:
train_df['Title'] = train_df['Title'].map(lambda x:1 if x in ['Miss', 'Mrs'] else 0)
train_df[['Title', 'Survived']].groupby(['Title'], as_index=False)\
.mean().sort_values(by='Survived', ascending=True)
Out[23]:
In [24]:
full_df['Title'] = full_df['Name'].map(title_split)
full_df['Title'] = full_df['Title'].map(lambda x:1 if x in ['Miss', 'Mrs'] else 0)
full_df.drop('Name', axis=1, inplace=True)
full_df.head()
Out[24]:
In [25]:
# sibsp+parch-->family
train_df['Family'] = train_df['Parch'] + train_df['SibSp'] + 1
train_df[['Family', 'Survived']].groupby(['Family'], as_index=False)\
.mean().sort_values(by='Family', ascending=True)
Out[25]:
In [26]:
# family-SibSp
train_df[['SibSp', 'Survived']].groupby(['SibSp'], as_index=False)\
.mean().sort_values(by='SibSp', ascending=True)
Out[26]:
In [27]:
# family-Parch
train_df[['Parch', 'Survived']].groupby(['Parch'], as_index=False)\
.mean().sort_values(by='Parch', ascending=True)
Out[27]:
In [28]:
# 备选方法：特征转换：使用特征转换器
#from sklearn.feature_extraction import DictVectorizer
#vec = DictVectorizer(sparse=False)
#full_df = vec.fit_transform(full_df.to_dict(orient='record'))
#print vec.feature_names_
五、调用机器学习模型¶
In [29]:
# 切分为原始训练集、预测集
len_train = len(train_df)
train_df = full_df.loc[:len_train-1, :]
predict_df = full_df.loc[len_train:, :]
print(train_df.shape, predict_df.shape)
In [30]:
# 将Survived值转换为int
train_df['Survived'] = train_df['Survived'].map(lambda x:int(x)).copy()
train_df.head()
Out[30]:
In [31]:
train_df['Sex'].value_counts()
Out[31]:
In [32]:
x = train_df.drop(['PassengerId', 'Survived'], axis=1)
y = train_df['Survived']
x_predict = predict_df.drop('Survived', axis=1)
x.shape, y.shape, x_predict.shape
Out[32]:
In [33]:
x.head()
Out[33]:
In [34]:
# 将原始训练集切分为训练集和测试集
from sklearn.model_selection import train_test_split
x_train, x_test, y_train, y_test = train_test_split(x,y,random_state=33,test_size=0.25)
In [35]:
# 调用模型，测评性能
# Logistics Regression、SGD、SVC均为参数模型，且需要标准化。
# 数据加工阶段，已使无需再标准化。
# 标准化方法
#from sklearn.preprocessing import StandardScaler
#ss = StandardScaler()
#x_train = ss.fit_transform(x_train)
#x_test = ss.transform(x_test)
# Logistics Regression
lr = LogisticRegression()
lr.fit(x_train, y_train)
acc_lr = lr.score(x_test, y_test)
# SGD
sgdc = SGDClassifier()
sgdc.fit(x_train, y_train)
acc_sgdc = sgdc.score(x_test, y_test)
# SVC
lsvc = LinearSVC()
lsvc.fit(x_train, y_train)
acc_lsvc = lsvc.score(x_test, y_test)
In [36]:
# 决策树、集合模型均为参数模型，但不需要标准化，但需要将类别特征向量化
# 数据加工阶段，已处理类别特征
# 特征向量化方法
#from sklearn.feature_extraction import DictVectorizer
#vec = DictVectorizer()
#x_train = vec.fit_transform(x_train.to_dict(orient='record'))
#x_test = vec.transform(x_test.to_dict(orient='record'))
# Tree
dtc = DecisionTreeClassifier()
dtc.fit(x_train, y_train)
acc_dtc = dtc.score(x_test, y_test)
# Ensemble-RandomForestClassifier
rfc = RandomForestClassifier()
rfc.fit(x_train, y_train)
acc_rfc = rfc.score(x_test, y_test)
# Ensemble-GradientBoostingClassifier
gbc = GradientBoostingClassifier()
gbc.fit(x_train, y_train)
acc_gbc = gbc.score(x_test, y_test)
In [37]:
# K-Neighbors：非参数模型，且需要标准化
knc = KNeighborsClassifier(n_neighbors=11)
knc.fit(x_train, y_train)
acc_knc = knc.score(x_test, y_test)
In [38]:
# rank of score
models = pd.DataFrame({'Model':['Logistics Regression',
'SGD',
'SVC',
'Tree',
'Ensemble-RandomForestClassifier',
'Ensemble-GradientBoostingClassifier',
'K-Neighbors'],
'Score':[acc_lr,
acc_sgdc,
acc_lsvc,
acc_dtc,
acc_rfc,
acc_gbc,
acc_knc]})
models.sort_values(by='Score', ascending=False)
Out[38]:
In [39]:
# 性能测评方法
# 混淆矩阵：精确度、召回率、二者的调和平均数
#from sklearn.metrics import classification_report
模型可视化：以决策树为例¶
In [42]:
# 小图
# from sklearn import tree
# tree.plot_tree(dtc.fit(x_train, y_train))
# 矢量图
# graphviz：用于输出可视化图片。首次安装后，可能需要添加至系统环境。
# import os
# os.environ['PATH'] = os.environ['PATH'] + (';c:\\Program Files (x86)\\Graphviz2.38\\bin\\')
import graphviz
from sklearn import tree
dot_data = tree.export_graphviz(dtc, out_file=None)
graph = graphviz.Source(dot_data)
graph.render("Titanic")
Out[42]:
In [43]:
>>> dot_data = tree.export_graphviz(dtc, out_file=None,
... feature_names=x_train.columns, # 特征名称列表
... class_names=y_train.name, # 类别标签名称
... filled=True, rounded=True,
... special_characters=True)
>>> graph = graphviz.Source(dot_data)
>>> graph
Out[43]:
六、结论¶
背景分析¶
- 本次目标属于分类问题，一般会选择监督学习中分类学习模型。
- 数据量未达到10万，因此暂不考虑模型效率问题。
特征分析¶
- 特征选择：Age Embarked Fare Parch PassengerId Pclass Sex SibSp Title
- 处理方式：对于每个特征，通过聚合数据，观察其与标签（幸存率）的关系，从而划分出级别，并用序数标记。最终都被整理为了离散值。
- 备选方式：抽了逐一手动处理特征，还可以考虑保留连续性变量，或调用特征向量化方法进行自动处理；但经过比较，发现逐一分析变量，手动聚合为离散值的方式，效果较好。
- 经验：
- 增加无关特征，准确率降低
- 增加特征title后，准确率
- 提高的：梯度上升决策树（+0.004），决策树（+0.004）
- 下降的：梯度下降（-0.09）
- 不变的：支持向量机，逻辑回归，随机森林，k邻近法（最优n由6变为11）
模型评价¶
准确率
- 参数模型：
- 最高，集合模型：梯度上升决策树；
- 其次，支持向量机，逻辑回归；
- 然后，随机森林，决策树；
- 最后，梯度下降。
- 非参数模型：当n取11时，准确率最高，且排名第二。
模型解释¶
- 相关系数矩阵：相关性较强的原始特征为sex、pclass、fare
- 决策树可视化：sex、pclass是最重要的特征，值得关注的人群有
- 对于男性：1等舱，20岁以上的，幸存率较好
- 对于男性：2、3等舱、无亲属、低票价的，幸存率较好
- 对于男性低龄儿童：2、3等舱、无亲属的，幸存率较差
- 对于男性低龄儿童：2、3等舱、既有直系亲属又有旁系亲属，幸存率较好
- 对于女性：1、2等舱的，生存率较差
- 对于女性：3等仓，上传地点为S，亲属较多的，幸存率较好
- 对于女性：3等仓，上传地点为S，低龄的，幸存率较好