Reinforcement learning-based optimal operation of ash deposit removal system to improve recycling efficiency of biomass for CO₂ reduction

Black liquor from pulp mills is valuable biomass that can be recycled as a CO₂-neutral, renewable fuel. However, biomass combustion produces significant ash deposits reducing the overall process efficiency. A recovery boiler generally uses an ash deposit removal system (ADRS), but ADRS operation is inefficient, and the recycling efficiency of the biomass is decreased, leading to an increase in CO₂ emission. This work proposed an optimal operation of ADRS to improve the recycling efficiency of biomass for CO₂ emission reduction based on reinforcement learning. The optimal operation of the ADRS was derived by the following steps. 1) Real-time process operating data (i.e., temperatures of the flue gas, water, and steam) were gathered and a computational fluid dynamics model was developed to predict the flue gas temperature in the superheater section. 2) The decrease in the heat transfer rate was calculated using the gathered data to define a reward update matrix. 3) A modified Q-learning algorithm was developed based on the defined reward update matrix, and the algorithm was used to derive the Q-matrix, a function that predicted the expected dynamic reward (i.e., priority for ash deposit removal) of performing a given action (i.e., sootblowing) at a given state (i.e., each sootblowing location). 4) Using the obtained Q-matrix, the optimal operating sequence was derived. As a result, 22.58 ton/d of black liquor was saved and the CO₂ emission decreased by 755–1390 ton/y with an increase in the net profit by $1,010,000.