Neurocontroller development for nonlinear processes utilising evolutionary reinforcement learning

[摘要] ENGLISH ABSTRACT:The growth in intelligent control has primarily been a reaction to the realisation thatnonlinear control theory has been unable to provide practical solutions to present daycontrol challenges. Consequently the chemical industry may be cited for numerousinstances of overdesign, which result as an attempt to avoiding operation near orwithin complex (often more economically viable) operating regimes. Within thesecomplex operating regimes robust control system performance may prove difficult toachieve using conventional (algorithmic) control methodologies.Biological neuronal control mechanisms demonstrate a remarkable ability to makeaccurate generalisations from sparse environmental information. Neural networks,with their ability to learn and their inherent massive parallel processing ability,introduce numerous opportunities for developing superior control structures forcomplex nonlinear systems. To facilitate neural network learning, reinforcementlearning techniques provide a framework which allows for learning from directinteractions with a dynamic environment. lts promise as a means of automating theknowledge acquisition process is beguiling, as it provides a means of developingcontrol strategies from cause and effect (reward and punishment) interactioninformation, without needing to specify how the goal is to be achieved.This study aims to establish evolutionary reinforcement learning as a powerful toolfor developing robust neurocontrollers for application in highly nonlinear processsystems. A novel evolutionary algorithm; Symbiotic, Adaptive Neuro-Evolution(SANE), is utilised to facilitate neurocontroller development. This study also aims tointroduce SANE as a means of integrating the process design and process controldevelopment functions, to obtain a single comprehensive calculation step formaximum economic benefit. This approach thus provides a tool with which to limitthe occurrence of overdesign in the process industry. To investigate the feasibility of evolutionary reinforcement learning in achievingthese aims, the SANE algorithm is implemented in an event-driven softwareenvironment (developed in Delphi 4.0), which may be applied for both simulation andreal world control problems. Four highly nonlinear reactor arrangements areconsidered in simulation studies. As a real world application, a novel batch distillationpilot plant, a Multi-Effect Batch Distillation (MEBAD) column, was constructed andcommissioned.The neurocontrollers developed using SANE in the complex simulation studies, werefound to exhibit excellent robustness and generalisation capabilities. In comparisonwith model predictive control implementations, the neurocontrollers proved far lesssensitive to model parameter uncertainties, removing the need for model mismatchcompensation to eliminate steady state off-set. The SANE algorithm also provedhighly effective in discovering the operating region of greatest economic return, whilesimultaneously developing a neurocontroller for this optimal operating point. SANE,however, demonstrated limited success in learning an effective control policy for theMEBAD pilot plant (poor generalisation), possibly due to limiting the algorithm'ssearch to a too small region of the state space and the disruptive effects of sensornoise on the evaluation process.For industrial applications, starting the evolutionary process from a random initialgenetic algorithm population may prove too costly in terms of time and financialconsiderations. Pretraining the genetic algorithm population on approximatesimulation models of the real process, may result in an acceptable search duration forthe optimal control policy. The application of this neurocontrol development approachfrom a plantwide perspective should also have significant benefits, as individualcontroller interactions are so doing implicitly eliminated.

[发布机构] Stellenbosch University