The MAT is essentially a different kind of smoothed moving average. It is made to filter out data sets that deviate from the specified absolute threshold and the result becomes a smoothing function. The goal here, inspired by time series analysis within mathematical study, is to eliminate data anomalies and generate a more accurate trendline.
Functionality:
This script calculates a filtered average by:
The filtered mean is then passed through a moving average function, where various types of moving averages like SMA, EMA, DEMA, TEMA, and ALMA can be applied. Some popular averages such as the HMA were omitted due to their heavy dependency on weighing specific data points.
Some information from "Time Series Analysis" regarding deviations
Approaches to Handle Anomalies?
Detection and Removal
Robust Statistics
Transformation
Functionality:
This script calculates a filtered average by:
- Determining the mean of the entire data series.
- Initializing sum and count variables.
- Iterating through the data to filter values that deviate from the mean beyond the threshold.
- Calculating a filtered mean based on the filtered data.
The filtered mean is then passed through a moving average function, where various types of moving averages like SMA, EMA, DEMA, TEMA, and ALMA can be applied. Some popular averages such as the HMA were omitted due to their heavy dependency on weighing specific data points.
Some information from "Time Series Analysis" regarding deviations
- Definition of Anomaly: An anomaly or outlier is a data point that differs significantly from other observations in the dataset. It can be caused by various reasons such as measurement errors, data entry errors, or genuine extreme observations.
- Impact on Mean: The mean (or average) of a dataset is calculated by summing all the values and dividing by the number of values. Since the mean is sensitive to extreme values, even a single outlier can significantly skew the mean.
- Example: Consider a simple time series dataset: . The value "150" is an anomaly in this context. If we calculate the mean with this outlier, it is (10 + 12 + 11 + 9 + 150) / 5 = 38.4. However, if we exclude the outlier, the mean becomes (10 + 12 + 11 + 9) / 4 = 10.5. The presence of the outlier has substantially increased the mean.
- Accuracy and Representativeness: While the mean calculated without outliers might be more "accurate" in the sense of being more representative of the central tendency of the bulk of the data, it's essential to note that anomalies might convey important information about the system being studied. Blindly removing or ignoring them might lead to overlooking significant events or phenomena.
Approaches to Handle Anomalies?
Detection and Removal
Robust Statistics
Transformation
版本注释:
Implemented ZLEMA option and updated plotting
KP