10.项目实战-交易数据异常检测

https://www.bilibili.com/video/av29439387

项目背景:信贷数据,是否是一笔正常、合理的贷款

方案:本case采用LR来作检测

1、样本不均衡解决方案:

主要有过采样、下采样

代码示例(下采样):

number_records_fraud = len(data[data.Class==1])
fraud_indeces = np.array(data[data.Class==1].index)

normal_indices = data[data.Class == 0].index

random_normal_indices = np.random.choice(normal_indices,number_records_fraud,replace=True)
random_normal_indices = np.array(random_normal_indices)

under_sample_indices = np.concatenate([fraud_indeces,random_normal_indices])

under_sample_dfata = data.iloc[under_sample_indices,:]

2、交叉验证和模型评估

train_test_split(),注意设置random_state

3、召回率和阈值:可以通过混淆矩阵来观察分布的变化

4、正则化:L1(权重系数的绝对值之和)和L2(权重系数的平方之和),可以遍历可能的C值来调整惩罚力度

5、过采样和SMOTE采样

from imblearn.over_sampling import SMOTE

oversample = SMOTE(random_state=0)

os_features,os_labels =  oversample.fit_sample(feature_train,label_train)

生成后的label为1的与label为0的数量一致

与下采样相比,过采样后得出的模型整体精度会高些,召回率会低点。建议采用过采样,数据越多越准确嘛

猜你喜欢

转载自blog.csdn.net/Gin077/article/details/82692945
10.