TensorFlow笔记-05-神经网络的实现过程（代码）

先回顾神经网络的实现过程

神经网络的实现过程

• 1.准备数据，提取特征，作为输入喂给神经网络
• 2.搭建NN结构，从输入到输出（先搭建计算图，再用会话执行）
  （NN前向传播算法===>计算输出）
• 3.大量特征数据喂给NN，迭代优化NN参数
  （NN反向传播算法===>优化参数训练模型）
• 4.使用训练好的模型，预测和分类
• 通过源代码，进一步理解 神经网络的实现过程
• 例子：随机产生32组生产出的零件的体积和重量，训练3000轮，每500轮输出一次损失函数
• 代码06文件：https://xpwi.github.io/py/TensorFlow/tf06.py

# coding:utf-8
# 导入模块，生成随机数据集
import tensorflow as tf
# 导入numpy模块，numpy是python的科学计算模块
import numpy as np
# 一次喂入神经网络多少组数据，数值不可以过大
BATCH_SIZE = 8
seed = 23455

# 基于seed产生随机数
rng = np.random.RandomState(seed)
# 随机数返回32行2列的矩阵 表示32组 体积和重量 作为输入数据集
X = rng.rand(32, 2)
# 从32行2列的矩阵中 去除一行判断如果和小于1 给Y赋值1 如果和不小于1 给Y赋值0
# 作为输入数据集的标签（正确答案）
Y = [[int(x0 + x1 <1)] for (x0, x1) in X]
print("X:\n", X)
print("Y:\n", Y)

# 定义神经网络的输入，参数和输出，定义前向传播过程
x = tf.placeholder(tf.float32, shape=(None, 2))
# y_即为0或1
y_ = tf.placeholder(tf.float32, shape=(None, 1))

# w1为2行3列
w1 = tf.Variable(tf.random_normal([2, 3], stddev=1, seed=1))
w2 = tf.Variable(tf.random_normal([3, 1], stddev=1, seed=1))

# matmul矩阵相乘
a = tf.matmul(x, w1)
y = tf.matmul(a, w2)

# 定义损失函数及反向传播方法
loss = tf.reduce_mean(tf.square(y-y_))
train_step = tf.train.GradientDescentOptimizer(0.001).minimize(loss)
# 其他优化方法
# train_step = tf.train.GMomentumOptimizer(0.001, 0.9).minimize(loss)
# train_step = tf.train.AdamOptimizer(0.001).minimize(loss)

# 生成会话，训练STEPS轮
with tf.Session() as sess:
    init_op = tf.global_variables_initializer()
    sess.run(init_op)
    # 输出目前（未经训练）的参数取值
    print("w1:\n", sess.run(w1))
    print("w2:\n", sess.run(w2))
    print("\n")

    # 训练模型3000轮
    STEPS = 3000
    for i in range(STEPS):
        start = (i*BATCH_SIZE) % 32
        end = start + BATCH_SIZE
        sess.run(train_step, feed_dict={x: X[start:end], y_: Y[start:end]})
        # 没500轮打印一次loss值
        if i % 500 == 0:
            total_loss = sess.run(loss, feed_dict={x: X, y_: Y})
            print("After %d training step(s), loss on all data is %g" %(i, total_loss))

    # 输出训练后的参数取值
    print("\n")
    print("w1:\n", sess.run(w1))
    print("w2:\n", sess.run(w2))

运行结果

X:
 [[0.83494319 0.11482951]
 [0.66899751 0.46594987]
 [0.60181666 0.58838408]
 [0.31836656 0.20502072]
 [0.87043944 0.02679395]
 [0.41539811 0.43938369]
 [0.68635684 0.24833404]
 [0.97315228 0.68541849]
 [0.03081617 0.89479913]
 [0.24665715 0.28584862]
 [0.31375667 0.47718349]
 [0.56689254 0.77079148]
 [0.7321604  0.35828963]
 [0.15724842 0.94294584]
 [0.34933722 0.84634483]
 [0.50304053 0.81299619]
 [0.23869886 0.9895604 ]
 [0.4636501  0.32531094]
 [0.36510487 0.97365522]
 [0.73350238 0.83833013]
 [0.61810158 0.12580353]
 [0.59274817 0.18779828]
 [0.87150299 0.34679501]
 [0.25883219 0.50002932]
 [0.75690948 0.83429824]
 [0.29316649 0.05646578]
 [0.10409134 0.88235166]
 [0.06727785 0.57784761]
 [0.38492705 0.48384792]
 [0.69234428 0.19687348]
 [0.42783492 0.73416985]
 [0.09696069 0.04883936]]
Y:
 [[1], [0], [0], [1], [1], [1], [1], [0], [1], [1], [1], [0], [0], [0], [0], [0], [0], [1], [0], [0], [1], [1], [0], [1], [0], [1], [1], [1], [1], [1], [0], [1]]
w1:
 [[-0.8113182   1.4845988   0.06532937]
 [-2.4427042   0.0992484   0.5912243 ]]
w2:
 [[-0.8113182 ]
 [ 1.4845988 ]
 [ 0.06532937]]

# 可以看到loss逐渐减小
After 0 training step(s), loss on all data is 5.13118
After 500 training step(s), loss on all data is 0.429111
After 1000 training step(s), loss on all data is 0.409789
After 1500 training step(s), loss on all data is 0.399923
After 2000 training step(s), loss on all data is 0.394146
After 2500 training step(s), loss on all data is 0.390597


w1:
 [[-0.7000663   0.91363174  0.0895357 ]
 [-2.3402493  -0.14641264  0.58823055]]
w2:
 [[-0.06024268]
 [ 0.91956186]
 [-0.06820709]]

运行结果分析

由神经网路的实现结果，我们可以看出，总共训练3000轮，每轮从X的数据集合Y的标签中抽取相对应的从start开始到end结束个特征值和标签，喂入神经网络，用sess，run求出loss，没500轮打印一次loss值，经过3000轮后我们打印出最终训练好的参数w1, w2

搭建神经网络的八股

• 搭建神经网络的八股：准备，前向传播，反向传播，迭代
• 以上面的代码为例
• (1)准备
  
  • 1.import
  • 2.常量定义
  • 3.生成数据集
• (2)前向传播: 定义输入，参数和输出
  
  • x =
  • y_ =
  • w1 =
  • w2 =
  • a =
  • y =
• (3)反向传播：定义损失函数，反向传播的方法
  
  • loss =
  • train_step =
• (4)在with中完成迭代，生成会话，训练STEPS轮

with tf.Session() as sess:
    init_op = tf.global_variables_initializer()
    sess.run(init_op)

    # 训练模型3000轮
    STEPS = 3000
    for i in range(STEPS):
        start = (i*BATCH_SIZE) % 32
        end = start + BATCH_SIZE
        sess.run(train_step, feed_dict={x: X[start:end], y_: Y[start:end]})
        # 没500轮打印一次loss值
        if i % 500 == 0:
            total_loss = sess.run(loss, feed_dict={x: X, y_: Y})
            print("After %d training step(s), loss on all data is %g" %(i, total_loss))

这样4步就可以实现神经网络的搭建了
拜拜