Enhanced Adaptive Confidence-Based Q Routing Algorithms For Network Traffic

• Main Author: Yap, Soon Teck
• Format: Thesis
• Language: English; English
• Published: 2004
• Online Access: http://psasir.upm.edu.my/id/eprint/581/1/549676_FSKTM_2004_11.pdf; http://psasir.upm.edu.my/id/eprint/581/

Two new adaptive routing algorithms named Enhanced Confidence-based Q (ECQ) and Enhanced Confidence-based Dual Reinforcement Q (ECDRQ) Routing Algorithms are proposed in this thesis. These two adaptive routing algorithms enhance the existing Confidence-based Q (CQ) and Confidence-based Dual Reinforcement Q (CDRQ) Routing Algorithms. In CQ Routing Algorithm, the confidence value (C value) can be used to improve the quality of exploration in Q Routing Algorithm. However, the C value incompletely evaluates how closely the Q value represents the current state of the network, which is measured in terms of estimated delivery time for a packet to arrive at its destination. An integrated solution for the above problem is the ECQ Routing Algorithm. ECQ Routing Algorithm is integrates the Variable of Decay Constant and Update All Q Value approaches for updating the C values of non-selected Q values. Using these C values would make those non-selected Q values more competitive in order to achieve updated and more reliable values. The CDRQ Routing Algorithm provides a solution to the problem addressed above by integrating the advantages of CQ Routing Algorithm and Dual Reinforcement Learning. The CQ Routing Algorithm is intended to improve the quality of actions made in exploration phase while dual reinforcement learning emphasises on increasing the number of actions occurred in exploration phase. However, the introduction of Confidence value and Backward Exploration may provide a solution to the problem stated above but it falls to another shortcoming known as the partially learning cycle problem, which is presented in this thesis. The ECDRQ Routing Algorithm integrates the ECQ and Dual Reinforcement Q (DRQ) Routing Algorithms with Alternative Q Value Approach to minimise the effect of partially learning cycle. By comparison, the proposed routing algorithms, ECQ and ECDRQ Routing Algorithms are more superior to its ancestors CQ and CDRQ Routing Algorithms in terms of average packet delivery time and average number of packets delivered. The ECDRQ and ECQ Routing Algorithms are tested against CDRQ and CQ Routing Algorithms respectively on an irregular 6 x 6 nodes network grid.