Control Logic Synthesis for Manufacturing Systems Using Markov Decision Processes

Changmin Lee, Jehyun Park, Jongeun Choi, Jaebok Ha, Sangyeong Lee

Research output: Contribution to journalConference articlepeer-review

1 Citation (Scopus)


Contemporary factory industry for mass production, (for example, manufacturing lines for liquid crystal display (LCD) panel display) deals with hundreds of products in the manufacturing and inspection processes simultaneously. Currently, how the product on a pallet moves in the manufacturing line is controlled via rule-based control logic programmed by human control logic designers. However, as the manufacturing system becomes larger, the complexity of the state space of the system increases exponentially for control logic designers, and the production rate of the manufacturing system significantly differs depending on the human logic designer. In this paper, we formulate a Markov Decision Process (MDP) model and synthesize the control logic for the linear motor-based manufacturing system, which will provide a consistent performance not depending on human logic designers. Our approach provides a fast re-design of the control logic when there are changes in the manufacturing systems as compared to rule-based control logic design by human logic designers. To solve a large-scale manufacturing system with high dimensional state spaces, we synthesize feasible and sub-optimal control logic, by modularizing the manufacturing system into multiple modules with manageable state space dimensions. To guarantee the safe operation without pallet collisions, we remove all the infeasible or collisional state-action pairs in the MDP modeling. We exhaustively simulate our control logic solution in a virtual manufacturing system for validation of our approach. Most importantly, we successfully validate our approach in the actual real-world test manufacturing system.

Original languageEnglish
Pages (from-to)495-502
Number of pages8
Issue number20
Publication statusPublished - 2021 Nov 1
Event2021 Modeling, Estimation and Control Conference, MECC 2021 - Austin, United States
Duration: 2021 Oct 242021 Oct 27

Bibliographical note

Publisher Copyright:
Copyright © 2021 The Authors. This is an open access article under the CC BY-NC-ND license

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering


Dive into the research topics of 'Control Logic Synthesis for Manufacturing Systems Using Markov Decision Processes'. Together they form a unique fingerprint.

Cite this