Automated quality control methods for sensor data: a novel observatory approach

[摘要] National and international networks and observatories of terrestrial-basedsensors are emerging rapidly. As such, there is demand for a standardizedapproach to data quality control, as well as interoperability of data amongsensor networks. The National Ecological Observatory Network (NEON) has begunconstructing their first terrestrial observing sites, with 60 locationsexpected to be distributed across the US by 2017. This will result in over14 000 automated sensors recording more than > 100 Tb of data per year.These data are then used to create other datasets and subsequent"higher-level" data products. In anticipation of this challenge, an overalldata quality assurance plan has been developed and the first suite of dataquality control measures defined. This data-driven approach focuses onautomated methods for defining a suite of plausibility test parameterthresholds. Specifically, these plausibility tests scrutinize the data rangeand variance of each measurement type by employing a suite of binary checks.The statistical basis for each of these tests is developed, and the methodsfor calculating test parameter thresholds are explored here. While thesetests have been used elsewhere, we apply them in a novel approach bycalculating their relevant test parameter thresholds. Finally, implementingautomated quality control is demonstrated with preliminary data from a NEONprototype site.

[学科分类] 地球化学与岩石