Hi, I am a senior Data Analyst and have been working with data and its anomalies all my career . Recently in my current company which is an utilities based one , i have seen the metering team struggling hard with the data discrepency
and data analmoly as they are dealing with miliions of rows in excel.
They do all the maths calculations to find if the voltage has seen a sudden drop or if current has gone too up too much. They use all the methods and formulaes in excel to derive this.
It has taken whole of their to fix the excel to identify one KPI such as reverse current.
This propelled me to think in building this ThresholdIQ, a browser based anomaly detection tool for operational data. I have made that users can drop excel/CSV/XML/JSON. It is extended Google Sheets Integration with the input data.
Everything is processed in Client side in browser, there is no backend server involved in processing of data except for storing static assets like email address for the subscription purpose.
ThresholdIQ detection engine works based 9 ML models as below
1. Multi window Z-score - Detect the spikes and drops based on 4 rolling window in parallel( W50,W100,W200,W500)
2. EWMA - Exponentially Weighted Moving Average , that identifies the drifts that multi window misses.
3. SARIMA - Seasonal Pattern Modelling. It raises deviations that is different from expected behaviour
4. Isolation forest Multivariate outlier dtection - Very useful methods that combines measures that are deviated.
5. Correlation deviation - It detects metrics that normaly go togther , but a sudden spike or drop together.
6. DBSCAN - Density based clustering- It raises to any points that doesnt fall in normal cluster.
7. Seasonal decomposition - Timeline analysis . It compares data against history for an hour.
8. Trend dtection - It identifies any failure that are so gradually moving towards a failure
9. Stuck/Zero Detection - It catches complete drop , for example in if voltage or current drops to 0.
i tested this on a 1,500-row AMI demo dataset — 60 raw metric alerts grouped down to 7 named incidents, with the voltage sag at 186V and the meter fault was correctly identified.
Technical stack
Cloudflare Pages + Workers + KV. Detection runs in a Web Worker for datasets above 1K rows so the UI stays responsive. The ML methods (EWMA, SARIMA, Isolation Forest, DBSCAN, correlation scoring) are loaded as ES modules and bundled into a blob URL at runtime to satisfy CSP without needing unsafe-eval.
The incident classification uses a priority-ordered rule chain — domain-specific named rules first, then generic statistical fallbacks (zero-from-nonzero, inverse correlation, systemic event).
The severity scoring combines the worst single-metric z-score with a breadth bonus for how many metrics deviated simultaneously — so a 41V voltage drop alone scores Emergency, and 3 metrics deviating together also scores Emergency even if each one individually would only be Critical
Honest limitations
Seasonal baseline is based on time-of-day grouping, not the STL decomposition - daily load data may still produce false positives especially on W50 window
We dont store data or data doesnt stay more then that session . If you refresh the page, data resets ( It is intentional for the Privacy Policy)
We dont have the live data connections for now. once this succeeds , the roadmap is to built this a data anomaly tool, that can be installed with all data connections.
Try it: thresholdiq.app (free, no account)
We have demo dataset to run and check how it works.
Looking for a honest feedback.
Hi, I am a senior Data Analyst and have been working with data and its anomalies all my career . Recently in my current company which is an utilities based one , i have seen the metering team struggling hard with the data discrepency and data analmoly as they are dealing with miliions of rows in excel.
They do all the maths calculations to find if the voltage has seen a sudden drop or if current has gone too up too much. They use all the methods and formulaes in excel to derive this.
It has taken whole of their to fix the excel to identify one KPI such as reverse current.
This propelled me to think in building this ThresholdIQ, a browser based anomaly detection tool for operational data. I have made that users can drop excel/CSV/XML/JSON. It is extended Google Sheets Integration with the input data.
Everything is processed in Client side in browser, there is no backend server involved in processing of data except for storing static assets like email address for the subscription purpose.
ThresholdIQ detection engine works based 9 ML models as below
1. Multi window Z-score - Detect the spikes and drops based on 4 rolling window in parallel( W50,W100,W200,W500)
2. EWMA - Exponentially Weighted Moving Average , that identifies the drifts that multi window misses.
3. SARIMA - Seasonal Pattern Modelling. It raises deviations that is different from expected behaviour
4. Isolation forest Multivariate outlier dtection - Very useful methods that combines measures that are deviated.
5. Correlation deviation - It detects metrics that normaly go togther , but a sudden spike or drop together.
6. DBSCAN - Density based clustering- It raises to any points that doesnt fall in normal cluster.
7. Seasonal decomposition - Timeline analysis . It compares data against history for an hour.
8. Trend dtection - It identifies any failure that are so gradually moving towards a failure
9. Stuck/Zero Detection - It catches complete drop , for example in if voltage or current drops to 0.
i tested this on a 1,500-row AMI demo dataset — 60 raw metric alerts grouped down to 7 named incidents, with the voltage sag at 186V and the meter fault was correctly identified.
Technical stack
Cloudflare Pages + Workers + KV. Detection runs in a Web Worker for datasets above 1K rows so the UI stays responsive. The ML methods (EWMA, SARIMA, Isolation Forest, DBSCAN, correlation scoring) are loaded as ES modules and bundled into a blob URL at runtime to satisfy CSP without needing unsafe-eval. The incident classification uses a priority-ordered rule chain — domain-specific named rules first, then generic statistical fallbacks (zero-from-nonzero, inverse correlation, systemic event).
The severity scoring combines the worst single-metric z-score with a breadth bonus for how many metrics deviated simultaneously — so a 41V voltage drop alone scores Emergency, and 3 metrics deviating together also scores Emergency even if each one individually would only be Critical
Honest limitations
Seasonal baseline is based on time-of-day grouping, not the STL decomposition - daily load data may still produce false positives especially on W50 window We dont store data or data doesnt stay more then that session . If you refresh the page, data resets ( It is intentional for the Privacy Policy) We dont have the live data connections for now. once this succeeds , the roadmap is to built this a data anomaly tool, that can be installed with all data connections.
Try it: thresholdiq.app (free, no account) We have demo dataset to run and check how it works. Looking for a honest feedback.