A neurocontrol paradigm for intelligent process control using evolutionary reinforcement learning

[摘要] ENGLISH ABSTRACT: A Neurocontrol Paradigm for Intelligent Process Control using EvolutionaryReinforcement LearningBalancing multiple business and operational objectives within a comprehensivecontrol strategy is a complex configuration task. Non-linearities and complex multipleprocess interactions combine as formidable cause-effect interrelationships. A clearunderstanding of these relationships is often instrumental to meeting the processcontrol objectives. However, such control system configurations are generallyconceived in a qualitative manner and with pronounced reliance on past effectiveconfigurations (Foss, 1973). Thirty years after Foss' critique, control systemconfiguration remains a largely heuristic affair.Biological methods of processing information are fundamentally different from themethods used in conventional control techniques. Biological neural mechanisms (i.e.,intelligent systems) are based on partial models, largely devoid of the system'sunderlying natural laws. Neural control strategies are carried out without a puremathematical formulation of the task or the environment. Rather, biological systemsrely on knowledge of cause-effect interactions, creating robust control strategies fromill-defined dynamic systems.Dynamic modelling may be either phenomenological or empirical. Phenomenologicalmodels are derived from first principles and typically consist of algebraic anddifferential equations. First principles modelling is both time consuming andexpensive. Vast data warehouses of historical plant data make empirical modellingattractive. Singular spectrum analysis (SSA) is a rapid model development techniquefor identifying dominant state variables from historical plant time series data. Sincetime series data invariably covers a limited region of the state space, SSA models arealmost necessarily partial models.Interpreting and learning causal relationships from dynamic models requires sufficientfeedback of the environment's state. Systemisation of the learning task is imperative.Reinforcement learning is a computational approach to understanding and automatinggoal-directed learning. This thesis aimed to establish a neurocontrol paradigm fornon-linear, high dimensional processes within an evolutionary reinforcement learning(ERL) framework. Symbiotic memetic neuro-evolution (SMNE) is an ERL algorithmdeveloped for global tuning of neurocontroller weights. SMNE is comprised of asymbiotic evolutionary algorithm and local particle swarm optimisation. Implicitfitness sharing ensures a global search and the synergy between global and localsearch speeds convergence.Several simulation studies have been undertaken, viz. a highly non-linear bioreactor, arigorous ball mill grinding circuit and the Tennessee Eastman control challenge.Pseudo-empirical modelling of an industrial fed-batch fermentation shows theapplication of SSA for developing partial models. Using SSA, state estimation isforthcoming without resorting to fundamental models. A dynamic model of a multieffectbatch distillation (MEBAD) pilot plant was fashioned using SSA. Thereafter,SMNE developed a neurocontroller for on-line implementation using the SSA modelof the MEBAD pilot plant.Both simulated and experimental studies confirmed the robust performance of ERLneurocontrollers. Coordinated flow sheet design, steady state optimisation and nonlinearcontroller development encompass a comprehensive methodology. Effectiveselection of controlled variables and pairing of process and manipulated variableswere implicit to the SMNE methodology. High economic performance was attained inhighly non-linear regions of the state space. SMNE imparted significant generalisationin the face of process uncertainty. Nevertheless, changing process conditions maynecessitate neurocontroller adaptation. Adaptive neural swarming (ANS) allows foradaptation to drifting process conditions and tracking of the economic optimum online.Additionally, SMNE allows for control strategy design beyond single unitoperations. SMNE is equally applicable to processes with high dimensionality,developing plant-wide control strategies. Many of the difficulties in conventionalplant-wide control may be circumvented in the biologically motivated approach of theSMNE algorithm. Future work will focus on refinements to both SMNE and SSA.SMNE and SSA thus offer a non-heuristic, quantitative approach that requiresminimal engineering judgement or knowledge, making the methodology free ofsubjective design input. Evolutionary reinforcement learning offers significantadvantages for developing high performance control strategies for the chemical,mineral and metallurgical industries. Symbiotic memetic neuro-evolution (SMNE),adaptive neural swarming (ANS) and singular spectrum analysis (SSA) present aresponse to Foss' critique.

[发布机构] Stellenbosch University