高级检索

具备安全机制的高效强化学习工业过程控制方法

Efficient Reinforcement Learning with Safety Mechanism for Industrial Process Control

  • 摘要: 传统强化学习方法在训练早期容易因随机探索触发危险状态,限制了其在具有强耦合、强非线性和严格安全约束等特点的工业过程控制中的应用。安全强化学习通过约束策略探索、修正高风险动作或引入安全先验,能够在一定程度上降低强化学习训练和部署过程中的运行风险。本文针对乙二醇精馏塔多变量连续控制问题,提出一种具备安全机制的多智能体连续动态策略规划方法。该方法将再沸器蒸汽流量、塔顶回流流量、塔釜采出流量、侧线采出流量和侧线冷却后温度建模为多个协同智能体动作,引入基于塔釜液位和乙二醇纯度的安全机制,并将原始策略动作和安全修正后的实际执行动作共同纳入经验回放池,在集中训练分散执行框架下利用相对熵正则化和安全干预信息对后续策略更新进行约束。乙二醇精馏塔仿真实验表明,训练40 d后,与人工操作相比,该方法使产品产出速率提升5.7%,平均净利润提升48.68%,且产品纯度始终满足质量约束。本文所提方法能够在化工过程控制中同时提高经济收益和改善学习稳定性,并在训练早期保障随机探索过程中的运行安全。

     

    Abstract: Conventional reinforcement learning methods are prone to triggering dangerous states due to random exploration in the early training stage, which limits their application to industrial process control characterized by strong coupling, strong nonlinearity, and strict safety constraints. Safe reinforcement learning can reduce operational risks during reinforcement learning training and deployment to some extent by constraining policy exploration, correcting high-risk actions, or incorporating safety priors. This paper proposes a multi-agent continuous dynamic strategy planning method with a safety mechanism for the multivariable continuous control problem of an ethylene glycol distillation column. In this method, the reboiler steam flow rate, overhead reflux flow rate, bottom product draw flow rate, side draw flow rate, and post-cooling side draw temperature are modeled as actions of multiple cooperative agents. A safety mechanism based on the column bottom level and ethylene glycol purity is introduced. Both the original policy actions and the safety-corrected executed actions are stored in the replay buffer. Under the centralized training with decentralized execution framework, relative entropy regularization and safety intervention information are used to constrain subsequent policy updates. Simulation results on the ethylene glycol distillation column show that, after 40 days of training, the proposed method increases product output by 5.72% and average net profit by 49.26% compared with human operation, while maintaining product purity within the quality constraint. The proposed method improves economic benefits and learning stability in chemical process control, while ensuring operational safety during early-stage random exploration.

     

/

返回文章
返回