Control Logic Synthesis for Manufacturing Systems Using Markov Decision Processes

Contemporary factory industry for mass production, (for example, manufacturing lines for liquid crystal display (LCD) panel display) deals with hundreds of products in the manufacturing and inspection processes simultaneously. Currently, how the product on a pallet moves in the manufacturing line is controlled via rule-based control logic programmed by human control logic designers. However, as the manufacturing system becomes larger, the complexity of the state space of the system increases exponentially for control logic designers, and the production rate of the manufacturing system significantly differs depending on the human logic designer. In this paper, we formulate a Markov Decision Process (MDP) model and synthesize the control logic for the linear motor-based manufacturing system, which will provide a consistent performance not depending on human logic designers. Our approach provides a fast re-design of the control logic when there are changes in the manufacturing systems as compared to rule-based control logic design by human logic designers. To solve a large-scale manufacturing system with high dimensional state spaces, we synthesize feasible and sub-optimal control logic, by modularizing the manufacturing system into multiple modules with manageable state space dimensions. To guarantee the safe operation without pallet collisions, we remove all the infeasible or collisional state-action pairs in the MDP modeling. We exhaustively simulate our control logic solution in a virtual manufacturing system for validation of our approach. Most importantly, we successfully validate our approach in the actual real-world test manufacturing system.