Abstract：

• 问题： The inclusion of artificial intelligence (AI) can improve the performance of routing protocols. Nowadays the application of AI over routing protocols is only applied to real devices, especially in wireless sensor nodes.

• 解决：In this paper, we present a new proposal to implement an intelligent routing protocol in a SDN topology. The intelligent routing protocol is based on the reinforcement learning process that allows choosing the best data transmission paths according to the best criteria and based on the network status.

强化学习：

根据维基百科的描述，强化学习定义如下:

强化学习是机器学习中的一个领域，强调如何基于环境而行动，以取得最大化的预期利益。其灵感来源于心理学中的行为主义理论，即有机体如何在环境给予的奖励或惩罚的刺激下，逐步形成对刺激的预期，产生能获得最大利益的习惯性行为。

在强化学习的世界里， 算法称之为Agent， 它与环境发生交互，Agent从环境中获取状态（state），并决定自己要做出的动作（action）.环境会根据自身的逻辑给Agent予以奖励（reward）。奖励有正向和反向之分。比如在游戏中，每击中一个敌人就是正向的奖励，掉血或者游戏结束就是反向的奖励。

来自掘金·腾讯云加社区

IDEA

• Quagga : it is open-source so it can be modified to add the AI-based improvements to the routing protocol. However, Quagga has the disadvantage that only allows having one router per PC.

• The algorithm showed in Fig. 2 provides a reinforcement learning method to establish a path between a source and a destination. Given a set of possible paths I and a set of network measurements (delay, loss rate and bandwidth) called M, the agent calculate for every available path i a cost ci by assigning different weights w1, w2 and w3 to the parameters. The data transmission is sent by the path with less cost.

• After spending some time, the following node in the path gives to the agent a reward d which is calculated using network parameters about the transmission through the path and a transfer function. This reward is the parameter that the agent (the router) wants to increase as much as possible. With that feedback from the paths, the agent adjusts again the weights trying to obtain a greater reward from a better path. This is the learning process in which the agent learns to choose the most important parameters to take account in the routing process.

• To execute this algorithm, the different elements present in the network need to exchange a set of messages. Fig. 3 shows the message exchange.

做法

• we have presented a new distributed routing proposal implemented over SDN. We have analyzed the way of building the SDN topology that runs routing protocol in a distributed way. Moreover, we have designed an intelligent algorithm based on reinforcement learning to improve some aspects of routing.

• This algorithm is added to the OSPF routing protocol using Quagga which allows modifying the routing algorithms.

效果：

• The results showthat the routing proposal works properly and it reaches better QoS features than the traditional one. Our proposal clearly achieves a more stable route with less loss rate that implies to have lower delay. In addition, the jitter obtained with the proposal is significantly better than the values offered by the traditional routing.