Fault tolerant container-based message queuing telemetry transport (MQTT) embedded cluster system

• Main Author: Thean, Zhong Ying
• Format: Final Year Project / Dissertation / Thesis
• Published: 2020
• Subjects: T Technology (General); TA Engineering (General). Civil engineering (General)
• Online Access: http://eprints.utar.edu.my/4401/1/18AGM05810_DIS.pdf; http://eprints.utar.edu.my/4401/

This dissertation work presents implementations of a distributed MQTT broker cluster in an edge-based environment. Since a single node broker can lose messages to the clients or the cloud when the node crashes. Hence, the purpose of this work to implement local fault tolerance to preserve the distributed system locally at the edge of network. Many previous studies have focused on distributed publish/subscribe systems but very few of them addressed the issue of local fault tolerance and the MQTT standard. Due to the recent popularity of the MQTT protocol, the MQTT middleware layer is developed to facilitate the cooperation of MQTT brokers without modifying the MQTT broker software. Also, the use of single-board computers as an edgebased hosting infrastructure keeps the cost low and can be flexibly sized according to workload demand and location of deployment. The purpose of the edge provisioning of the broker cluster is to reduce end-to-end latency for IoT and M2M streaming applications. The proposed system uses two approaches to realize fault tolerance. First, the proposed system tolerates node crashes by maintaining consistency of state information using time-to-live (TTL) subscription routing entries. Next, message loss is corrected through retransmission at the broker nodes to the subscribers. The evaluations demonstrated improved scalability for the horizontal scaling approach and successful recovery of failed publication during failover. The worst-case end-to-end latency of the proposed system is at a maximum of 42 milliseconds. All missed publications are redelivered to the subscriber during failover without significant delay between the retransmitted messages. The jitter values between recovered messages during the recovery period range from 10 to 20 milliseconds. The maximum recovery time of the proposed broker cluster is at least 256.33 milliseconds, which is within hundreds of milliseconds difference, compared to 50 milliseconds of the primary-backup broker approach. The fail-test confirms the reliability of the MQTT cluster, as failed publications can be redelivered during broker failure. The evaluations demonstrated the feasibility of the proposed broker cluster to maintain consistent latencies and support reliable MQTT services despite server failures.