之前我们其实已经写过了,这次既然原文单独列了一章,我们也来再写一次好了。原文希望我们把RMSE压到110以下,我这里验证集的RMSE 115左右,因随机数种子最低降到过112左右,在设定好种子的情况下seed(1) 验证集的rmse在116左右。
上次我们用xs1,xs2,xs3...来得到特征集的tensor,如若columns太多,操作起来很繁琐,这次我把tensor压到了一个列表里,然后通过concat来组合。
import tensorflow as tf
import numpy as np
import pandas as pd
from tensorflow.data import Dataset
df = pd.read_csv('california_housing_train.csv')
df['per_rooms'] = df['total_rooms'] / df['population']
df['median_house_value'] /= 1000
df = df.reindex(np.random.permutation(df.index))
df = df.sort_index()
tf.set_random_seed(1)
def train_validation(df):
features = df[['longitude', 'latitude', 'housing_median_age',
'total_rooms', 'total_bedrooms', 'population', 'households',
'median_income', 'per_rooms']]
targets = df['median_house_value']
train_features = (features.head(12000).astype('float32')).copy()
validation_features = (features.tail(5000).astype('float32')).copy()
train_targets = (targets.head(12000).astype('float32')).copy()
validation_targets = (targets.tail(5000).astype('float32')).copy()
return train_features, train_targets, validation_features, validation_targets
def get_features(xs_dict):
feautres = list(xs_dict.values())
outputs = []
for i in feautres:
outputs.append(tf.expand_dims(i, -1))
outputs = tf.concat([i for i in outputs], -1)
return outputs
def my_input_fn(features, labels, batch_size=1, num_epochs=1, shuffle=False):
features = {key: value for key, value in features.items()}
ds = Dataset.from_tensor_slices((features, labels))
ds = ds.batch(batch_size).repeat(num_epochs)
if shuffle:
ds.shuffle(10000)
features, labels = ds.make_one_shot_iterator().get_next()
return features, labels
def add_layer(inputs, input_size, output_size, activation_function=None):
weights = tf.Variable(tf.random_normal([input_size, output_size], stddev=0.1))
tf.add_to_collection('losses', tf.contrib.layers.l2_regularizer(0.001)(weights))
biases = tf.Variable(tf.zeros(output_size) + 0.1)
wx_b = tf.matmul(inputs, weights) + biases
if activation_function is None:
outputs = wx_b
else:
outputs = activation_function(wx_b)
return weights, biases, outputs
def loss(pred, ys, regularizer=True):
rmse = tf.sqrt(tf.reduce_mean(tf.square(ys - pred)))
if regularizer is True:
loss = rmse + tf.add_n(tf.get_collection('losses'))
else:
loss = rmse
return loss
def train_step(learning_rate, loss):
train_step = tf.train.AdamOptimizer(learning_rate).minimize(loss)
return train_step
xs, ys, dx, dy =train_validation(df)
xs, ys = my_input_fn(xs, ys, batch_size=1000, num_epochs=300)
dx, dy = my_input_fn(dx, dy, batch_size=5000, num_epochs=40)
x_input = get_features(xs)
x_validation = get_features(dx)
w1, b1, l1 = add_layer(x_input, 9, 30, activation_function=tf.nn.tanh)
w2, b2, l2 = add_layer(l1, 30, 6)
w3, b3, l3 = add_layer(l2, 6, 1)
train_loss = loss(l3, ys, regularizer=True)
train = train_step(0.005, train_loss)
sess = tf.Session()
init = tf.global_variables_initializer()
sess.run(init)
for i in range(2000):
sess.run(train)
if i % 50 == 0:
pred_l1 = tf.nn.tanh(tf.matmul(x_validation, w1) + b1)
pred_l2 = tf.matmul(pred_l1, w2) + b2
pred = tf.matmul(pred_l2, w3) + b3
pred_loss = loss(pred, dy, regularizer=False)
total_loss = sess.run([train_loss, pred_loss])
print('train loss is :', '%0.2f' % total_loss[0], 'validation loss is:', '%0.2f' % total_loss[1])
这里我只调整了第一层和第二层的weights,learning_rate, 优化器,batch_size还有正则化率。当然还有很多参数可以调整,我们不妨自己动手试试,这里我之所以把seed固定,是因为我希望结果的变化是因为我更改了某一参数而变化,而不是因为初始weights的影响。在调整的时候请不要对特征进行操作,本次的目的不是特征工程。而是在输入特征固定的情况下,调整参数来看神经网络的变化。这里我没有添加太多的隐藏层,你们也可以试试添加隐藏层来看看是否可以得到更好的模型。