Reinforcement learning for the control of traffic flow on highways

[摘要] ENGLISH ABSTRACT: Traffc congestion has become a significant problem around the world, not only in first-worldcountries, but also in third-world countries such as South Africa. Due to spatial limitations, especially in well-developed metropolitan areas, which typically experience the worst congestion problems, capacity expansion is not always feasible for relieving the pressure on the transportationnetwork. Furthermore, the theory of induced traffic demand suggests that increasing highway capacity is not a long-term solution to traffic congestion due to additional traffic demandon new or updated routes, induced by commuters' perception that new or upgraded routes should be congestion free. As a result, various approaches toward improving highway traffic flow without increasing infrastructure capacity have been proposed in the literature.Ramp metering and variable speed limits are the best-known control measures for effective traffic flow on highways. In most approaches towards solving the control problems presented by these control measures, optimal control techniques or online feedback control have been employed.Feedback control does not, however, guarantee optimality with respect to the on-ramp meteringrate or the speed limit chosen, while optimal control techniques are limited to small networksdue to their large computational burden.Reinforcement learning is a promising alternative, providing the means and framework requiredto achieve near-optimal control policies at a fraction of the computational burden associatedwith optimal control algorithms. In this dissertation, a decentralised reinforcement learningapproach is adopted towards simultaneously solving both the ramp metering and variable speedlimit control problems.The dawn of the autononomous vehicle promises further improvements in traffic flow whichmay be achieved over and above those of the aforementioned established highway traffic controlmeasures, if their capabilities are harnessed effectively. A novel method of ramp meteringby autonomous vehicles is introduced in this dissertation, based on the premise that specificinstructions may be provided to autonomus vehicles travelling along an on-ramp. The control problem presented by this method of ramp metering via autonomous vehicles is also solved using a reinforcement learning approach.The above solution approaches are implemented as a concept demonstrator within a simple,benchmark microscopic highway traffic simulation model. The effectiveness of the decentralisedreinforcement learning approach is evaluated by means of statistical comparisons within the context of this simple benchmark simulation model. These approaches are finally applied within the context of a real-world case study simulation model of a section of the N1 highway outboundout of Cape Town, South Africa in order to demonstrate the effectiveness of the approacheswithin the context of a realistic scenario based on a real highway network and real traffic flow data.

[发布机构] Stellenbosch University