Acquisition of context-based word recognition by reinforcement learning using a recurrent neural network

• Main Author: Ahmad Afif, Mohd Faudzi
• Format: Thesis
• Language: English
• Published: 2012
• Subjects: Q Science (General)
• Online Access: http://umpir.ump.edu.my/id/eprint/3569/1/AHMAD_AFIF_MOHD_FAUDZI.PDF; http://umpir.ump.edu.my/id/eprint/3569/

The eye movement and recognition in humans seem very flexible and intelligent. The flexible and intelligent recognition is not only depending on the informatiOn that belonged by target object, but also is supported by other information, including past knowledge and contextual information. For an example, when we read a book, we do not usually seem to read it -by recognizing each character one by one. We would predict a word from the first two or several characters, and also would utilize the story context to expect the next character or word. In order to make flexible recognition, we consider many things simultaneously, and move our eyes, recognize and understand the context flexibly. Such parallel consideration and flexibility must be achieved by our parallel and flexible brain, and learning plays an important role in, it. In our laboratory, the coupling of a neural network (NN) and reinforcement learning (RL) is considered useful because of its autonomous, parallel and flexible learning ability. In the previous research, it was verified through simulations and also using a real camera that the appropriate camera motion, recognition and recognition timing were successfully acquired. However, since a regular layered neural network was used, the recognition and movement functions were limited to the case where the whole pattern is in the visual field. Furthermore, only two patterns were used. In this thesis, context-based word recognition learning system was developed. 6 words that need context-based recognition function for the words to be recognized were chosen. The learning system was trained to recognize all the chosen words. As a learning method, the combination of Reinforcement Learning and a Recurrent Neural Network (RNN) was applied. The developed learning system has a 4-layered RNN and it was trained by BPTT method based on teaching signal that was generated by Q-Learning algorithm. The learning system was trained on several tasks using simulations or real time learning in order to verify whether flexible recognition could emerge through this context-based word recognition learning. There are two types of problem tasks in this thesis. The first type using ideal- images as learning data and the second one using real camera captured images. For the first type, two tasks were done through simulations. The one is a fixed initial position task, and the second one is a random initial position task. As results, for both simulations the system manages to recognize all the prepared words. Here, the relation between the parameter of discount factor teaching signal was also discussed and how to choose the appropriate settings was proposed. In the second type of task, both simulations were trained using real camera captured images that were prepared beforehand as samples. After the learning was successfully verified, finally, the system was trained for the same task in real time. After the real time learning, learning was successful and system manages to recognize the entire prepared patterns. All these results show that by applying a combination of Reinforcement Learning and a Recurrent Neural Network learning method, the context-based word recognition can be achieved. Flexible recognition function in the appropriate timing is mostly acquired.