Reinforcement learning-based optimal operation of ash deposit removal system to improve recycling efficiency of biomass for CO2 reduction

Jonghun Lim, Hyungtae Cho, Hyukwon Kwon, Hyundo Park, Junghwan Kim

Research output: Contribution to journalArticlepeer-review


Black liquor from pulp mills is valuable biomass that can be recycled as a CO2-neutral, renewable fuel. However, biomass combustion produces significant ash deposits reducing the overall process efficiency. A recovery boiler generally uses an ash deposit removal system (ADRS), but ADRS operation is inefficient, and the recycling efficiency of the biomass is decreased, leading to an increase in CO2 emission. This work proposed an optimal operation of ADRS to improve the recycling efficiency of biomass for CO2 emission reduction based on reinforcement learning. The optimal operation of the ADRS was derived by the following steps. 1) Real-time process operating data (i.e., temperatures of the flue gas, water, and steam) were gathered and a computational fluid dynamics model was developed to predict the flue gas temperature in the superheater section. 2) The decrease in the heat transfer rate was calculated using the gathered data to define a reward update matrix. 3) A modified Q-learning algorithm was developed based on the defined reward update matrix, and the algorithm was used to derive the Q-matrix, a function that predicted the expected dynamic reward (i.e., priority for ash deposit removal) of performing a given action (i.e., sootblowing) at a given state (i.e., each sootblowing location). 4) Using the obtained Q-matrix, the optimal operating sequence was derived. As a result, 22.58 ton/d of black liquor was saved and the CO2 emission decreased by 755–1390 ton/y with an increase in the net profit by $1,010,000.

Original languageEnglish
Article number133605
JournalJournal of Cleaner Production
Publication statusPublished - 2022 Oct 10

Bibliographical note

Funding Information:
This work was supported by the Korean Institute of Industrial Technology within the framework of the following projects: “Development of Global Optimization System for Energy Process [grant number IR-22-0040 , IZ-22-0049 , UR-22-0031 ]”, “Development of AI Platform for Continuous Manufacturing of Chemical Process [grant number JH-22-0004 ]” and “Development of hybrid model and software to optimization of ash removal system in recovery boiler for power generation [grant number JH-21-0006 ]”.

Publisher Copyright:
© 2022 The Authors

All Science Journal Classification (ASJC) codes

  • Renewable Energy, Sustainability and the Environment
  • Building and Construction
  • Environmental Science(all)
  • Strategy and Management
  • Industrial and Manufacturing Engineering


Dive into the research topics of 'Reinforcement learning-based optimal operation of ash deposit removal system to improve recycling efficiency of biomass for CO2 reduction'. Together they form a unique fingerprint.

Cite this