Reinforcement Learning : based SDN Switch Migration with Dynamic Weight of Controller Resources
- 주제(키워드) Software-defined networking , Load-balancing , SDN , Reinforcement learning
- 주제(DDC) 006.31
- 발행기관 아주대학교
- 지도교수 Sangyoon Oh
- 발행년도 2021
- 학위수여년월 2021. 8
- 학위명 석사
- 학과 및 전공 일반대학원 인공지능학과
- 실제URI http://www.dcollection.net/handler/ajou/000000031066
- 본문언어 영어
- 저작권 아주대학교 논문은 저작권에 의해 보호받습니다.
초록/요약
Software defined networking (SDN) provides programmable to the networking devices, which makes the network management to be easier than traditional networking paradigm. The distributed controller in SDN is a promising approach to scale the network, and it provides the better quality of experience in today’s emerging internet. In SDN, switches are managed by their associated controller and the load of the controllers are changing according to the flow requests from the switches. As a consequence, uneven load distribution among the controllers become the problem among the SDN controllers. Dynamic migration of switches from a controller with overutilization to another underutilized controller can balance the loads between the controllers, which is termed as switch migration (SM) for the balanced load distribution among the SDN controllers. Several SM approaches have been studied with iterative optimization schemes to select the optimal mapping of switch and controller for the migration. Reinforcement Learning (RL) also has been adopted for the intelligence decision-making in SM domain. However, these schemes assume a limited number of cases in migration action selection. Subsequently, it is difficult for RL agents to finely tune the action selection based on the current state. In the SM domain, such limitation of migration cases is hard to provide the optimal switch-to-controller pair in the training process. Therefore, we proposed a DRL-based SM scheme, NAF-LB, with dynamic weights in the utilization model for the decision making of SM. With NAF-LB, the detection of load imbalance status of the SDN controllers becomes optimized, and the switch can choose more optimal controller from the candidate controller set. To train the SM problem with fine-grained action selection, we utilize the normalized advantage function (NAF). We made the verification for the performance of the proposed algorithm in simulated SDN environments, and the results showed that NAF-LB can detect the utilization status of each resource utilization along with the better load distribution among the controllers. Our proposed scheme improves the load-balancing results by up to 10%–11% over the comparative scheme with coarse-grained action selection in the simulated scenarios.
more목차
1. Introduction 1
2. Related works 8
2.1 Conventional switch migration schemes 8
2.2 Reinforcement Learning based schemes 12
3. System model 14
3.1 Problem formulation 14
3.2 Normalized Advantage Function model formulation 18
4. Proposed Reinforcement Learning based switch migration scheme 20
4.1 Design of NAF-LB 20
4.2 Proposed Switch Migration algorithm 23
5. Evaluation 25
5.1 Experimental environment 25
5.2 Experimental results and discussion 28
5.2.1 Load balancing per resource dimension 28
5.2.2 Load balancing per controller 30
6. Conclusion and future work 33
References 35
