A distributed rule-based expert system for large event stream processing

[摘要] Rule-based expert systems (RBSs) provide an efficient solution to many problems that involve event stream processing. With today’s needs to process larger streams, many approaches have been proposed to distribute the rule engines behind RBSs. However, there are some issues which limit the potential of distributed RBSs in the current big data era, such as the load imbalance due to their distribution methods, and low parallelism originated from the continuous operator model. To address these issues, we propose a new architecture for distributing rule engines. This architecture adopts the dynamic job assignment and the micro-batching strategies, which have recently arisen in the big data community, to remove the load imbalance and increase parallelism of distributed rule engines. An automated transformation framework based on Model-driven Architecture (MDA) is presented, which can be used to transform the current rule engines to work on the proposed architecture. This work is validated by a 2-step verification. In addition, we propose a generic benchmark for evaluating the performance of distributed rule engines. The performance of the proposed architecture is discussed and directions for future research are suggested. The contribution of this study can be viewed from two different angles: for the rule-based system community, this thesis documents an improvement to the rule engines by fully adopting big data technologies; for the big data community, it is an early proposal to process large event streams using a well crafted rule-based system. Our results show the proposed approach can benefit both research communities.

[发布机构] University:University of Birmingham;Department:School of Computer Science

[关键词] T Technology;T Technology (General)