A Better Way to Think About False Positives
‘True’ false positive alarms are rarely issued by the Qube system. This articles explains what a false positive alarm is in the Qube system, how to make sense of METEC results, and the best practices for responding to potential false positive alarms.
Author: Brad Roger, Product Manager at Qube Technologies
What is a false positive alarm?
A false positive alarm is an emission event that is recorded in the system when methane cannot be confirmed in the field. ‘True’ false positive events rarely occur in the Qube system. If the system is detecting a leak, there is almost certainly something there.
But depending on how broadly a false positive emission event is defined - and the manner in which it is determined - there are instances when it may appear that Qube is producing a true false positive alarm.
How METEC determines if a false positive occured
The METEC protocol relies on monitoring technologies to provide individual emission reports that describe each distinct emission event with a single source, and the start and end time. Crucially, it also compares the number of emission reports to the number of real emissions released on site.
This definition does not consider whether there is an active emission when the report is submitted, and whether the predicted volumes over the timespan are correct. Instead, METEC focuses on the number of reports issued.
Due to the nature of continuous monitoring, a gap in observability or a small change in localization can cause emissions splitting.
Emission splitting occurs when Qube produces more than one emission event that describes a single emission.
The predicted volume over the emission timespan may be exactly right, but splitting it into two distinct events is ‘penalized’ in the current METEC protocol.
The following table illustrates a hypothetical case where Qube predicts an emission event with the overall time of the emission properly bounded, but the emission is split into three discrete events because the plume went a sufficient duration without passing over a device:
An abstract illustration of METEC results compared to Qube’s expression of results
This table underscores the reason we do not produce alarms off of emission events alone. Instead, we issue alarms based on the cumulative volume released over a given period, which is much more indicative of the severity of a particular event.
Whether emission splitting occurs or not has no impact on users of our system. Correctly determining the quantification and localization of methane emissions, and doing so in a fast and intuitive way, is much more important to our users than defining the distinct time bounds of an emission. This is why Qube prioritizes the precision of quantification and localization metrics.
What If you receive an alarm but don’t find a leak?
As noted above, ‘true’ false positive events rarely occur in the Qube system. If a response team cannot confirm an emission in the field, the reason can likely be attributed to one of the following factors:
1. Anomalous environmental conditions which mislead the localization algorithm. For example, if prevailing winds shift abruptly while the system is localizing an emission event, then a change in localization may occur.
2. Qube presents the ‘most likely' emission source from all potential emission sources on the site. Sometimes the most likely source identified by the system is equally probable to other proximate sources, but Qube shows only a single source.
3. The intermittency of the leak. For example, if the event was brief or related to other operations such as routine maintenance that has since been resolved by the time operations teams check out the alarm, the system may detect an emission that cannot be mapped to an active leak in the field.
Best practices for responding to alarm notifications
There are several best practices that operators can follow to prevent false positive interpretations from occurring, and minimize resources used to discover the cause.
1. Before actioning a response, review data in the Qube platform alongside all other available information:
What are real-time environmental conditions? Very high or very low wind speeds may impact localization and the ‘most likely’ source presented to the end user.
What is going on operationally at the moment that may cause or influence the detection of gas? Routine activities such as tank maintenance near a monitoring device can trigger a positive detection that is brief and not verifiable when checked by staff.
2. Evaluate your alarm thresholds. Are they set so low that non-actionable events are being captured as well? Establishing the right alarm parameters is essential for driving the appropriate response to leak events. (Learn more about best practices for setting alarm parameters in this article.)
If the potential causes and evaluative steps described above do not produce a verifiable explanation, or if the event is recurring, then a deeper dive is required with your Customer Success Representative.
Stay tuned for our next blog where we’ll discuss quantification and localization in the context of METEC.