Optimisation of fed-batch fermentation process using deep reinforcement learning

• Main Author: Chai, Wan Ying
• Format: Thesis
• Language: English; English
• Published: 2023
• Subjects: TP155-156 Chemical engineering
• Online Access: https://eprints.ums.edu.my/id/eprint/40554/1/24%20PAGES.pdf; https://eprints.ums.edu.my/id/eprint/40554/2/FULLTEXT.pdf; https://eprints.ums.edu.my/id/eprint/40554/

Fed-batch fermentation process has always been a challenge for optimisation because it is highly non-linear and complex. Deep reinforcement learning is a self-learning algorithm through trial and error and experience, without any prior knowledge. This research aimed to determine the optimal feeding strategy for fed-batch baker’s yeast fermentation process using the deep reinforcement learning algorithm in maximising the final production of yeast, while minimising the undesired ethanol formation. The kinetic and dynamic behaviour of the fed-batch baker’s yeast fermentation was simulated and modelled using MATLAB, with no experimental work carried out. The proposed deep reinforcement learning algorithm, which integrates an artificial neural network with traditional reinforcement learning, was formulated based on the optimisation objective by manipulating only the substrate feeding rate. The performance of the proposed algorithm was compared with a pre-determined exponential feeding profile and a genetic algorithm. Results for the nominal condition show that the proposed algorithm produced final yeast concentration 33.42 g/l and 6.02 g/l higher than the exponential feeding and genetic algorithm, respectively. At the same time, the total ethanol formation is 0.19 g/l and 0.03 g/l lower than the exponential feeding and genetic algorithm, respectively. In other cases of different initial yeast and substrate concentrations, the proposed algorithm in general outperforms the exponential feeding profile while produces comparable results to the genetic algorithm. When dealing with model mismatch (±15% parameter variation in critical growth and maximum glucose uptake rate) and process disturbance (±20% deviation in substrate feeding concentration), the proposed algorithm was able to handle the changes with a minor effect on the yeast yield up to 13.78% and 2.52%, respectively, across all different initial condition cases. In conclusion, a deep reinforcement learning algorithm was successfully developed for the substrate feeding rate optimisation in the fed-batch baker’s yeast fermentation process. The proposed algorithm improves the productivity of yeast while limiting ethanol formation and shows satisfactory performance in dealing with model mismatch and process disturbance.