  {"id":789,"date":"2021-12-29T14:03:43","date_gmt":"2021-12-29T14:03:43","guid":{"rendered":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/?p=789"},"modified":"2022-02-18T12:03:12","modified_gmt":"2022-02-18T12:03:12","slug":"report-anomaly-detection-for-functional-data","status":"publish","type":"post","link":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/2021\/12\/29\/report-anomaly-detection-for-functional-data\/","title":{"rendered":"Anomaly detection for functional data"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"789\" class=\"elementor elementor-789\">\n\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-d4c8008 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"d4c8008\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-eaafa1a\" data-id=\"eaafa1a\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-d223a75 elementor-widget elementor-widget-heading\" data-id=\"d223a75\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">1. Introduction<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-a0e99e7 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"a0e99e7\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-90f1a5c\" data-id=\"90f1a5c\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-2ae57ee elementor-widget elementor-widget-text-editor\" data-id=\"2ae57ee\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>We call an anomaly, an observation that differs significantly from the rest of the data. These are important to detect as they can indicate the existence of a problem. Early identification could even prevent the problem recurring and avoid subsequently worse issues. There exist different types of anomalies; it may be a single observation that deviates from the rest, which is called a point anomaly or a group of points called a collective anomaly. In this blog however, we will be looking at anomalies in functional data. Figure 1 shows the data used, it consists of measurements made minute by minute across 50 days but we do not know what it measures. Hence, the two main types of anomaly will be, shape anomalies, which are curves that display a different behaviour than the other curves and magnitude anomalies, curves that at some point have a significant spike away from the rest of the curves. To detect these anomalies, we will look at two different methods. The total variation depth and modified shape similarity index (TVDMSS) and the functional outlyingness map using directional outlyingness (FOM(DO)), which methodology is discussed in Section 2. Then, Section 3 contains the results obtained from applying these methods to the data described and compares them. Finally, Section 4 contains the limitations of both methods and a general conclusion. Throughout this blog, the terms anomaly and outliers will be used interchangeably.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-1f62824 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"1f62824\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-b683342\" data-id=\"b683342\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-288c240 elementor-widget elementor-widget-image\" data-id=\"288c240\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img fetchpriority=\"high\" decoding=\"async\" width=\"768\" height=\"461\" src=\"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figure1-768x461.png\" class=\"attachment-medium_large size-medium_large wp-image-1120\" alt=\"Figure1\" srcset=\"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figure1-768x461.png 768w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figure1-300x180.png 300w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figure1-1024x615.png 1024w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figure1.png 1323w\" sizes=\"(max-width: 768px) 100vw, 768px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-fd34f83 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"fd34f83\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-7451ec1\" data-id=\"7451ec1\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-44e8d3c elementor-widget elementor-widget-spacer\" data-id=\"44e8d3c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"spacer.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-spacer\">\n\t\t\t<div class=\"elementor-spacer-inner\"><\/div>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-182457e elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"182457e\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-1249e57\" data-id=\"1249e57\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-62872ee elementor-widget elementor-widget-heading\" data-id=\"62872ee\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">2. Methods<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-6b249f7 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"6b249f7\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-a10fba1\" data-id=\"a10fba1\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-82c287e elementor-widget elementor-widget-heading\" data-id=\"82c287e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">2.1 Total variation depth and modified shape similarity index<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-c3d7ee2 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"c3d7ee2\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-d8bb18e\" data-id=\"d8bb18e\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-6ff4864 elementor-widget elementor-widget-text-editor\" data-id=\"6ff4864\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Since we are working with functional data, standard outliers detection techniques (for a single time series) will not be applicable. Here, outliers will have to be detected with respect to each curve.\u00a0<span style=\"font-family: Poppins;font-size: 13px;font-style: normal;font-weight: 400;text-align: start\">The following TVDMSS method is based on the work of\u00a0<\/span><a style=\"font-family: Poppins;font-size: 13px;font-style: normal;font-weight: 400;text-align: start\" href=\"https:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/00401706.2019.1574241\" target=\"_blank\" rel=\"noopener\">Huang &amp; Sun (2019).<\/a><span style=\"font-family: Poppins;font-size: 13px;font-style: normal;font-weight: 400;text-align: start\">\u00a0<\/span><span style=\"font-size: 13px\">For each curves in our functional, data both the total variation depth (TVD) and modified shape similarity (MSS) are computed. The TVD can be interpreted as the total variability in the depth (along an interval of interest) of a curve compared to all other curves. At any given point in time, the deepest curve would be the one closest to the median of all curves at that point in time. The MSS is simply a measure of how similar the shape of a curve is compared to the shape of the all other curves. The lower the similarity the more likely it is to be considered an outlier.<\/span><\/p><p>Using these measures we can then start detecting anomalies. First, a classical boxplot is constructed using the MSS of each curve. An outlier corresponds to a curve for which the MSS is less than the P \u00d7 50% central region. Were the 50% central region corresponds to the 50% deepest curves. Outliers detected using this approach are considered shape outliers. Next, shape outliers are removed and a functional boxplot is constructed using the TVD. This functional boxplot is based on all the original curves prior to removing the shape outliers. Once again, the outlier detection region is Q \u00d7 50% central region. The RStudio function (<a href=\"https:\/\/rdrr.io\/cran\/fdaoutlier\/man\/tvdmss.html\" target=\"_blank\" rel=\"noopener\"><em>tvdmss<\/em><\/a> from the package <a href=\"https:\/\/cran.r-project.org\/web\/packages\/fdaoutlier\/fdaoutlier.pdf\" target=\"_blank\" rel=\"noopener\"><em>fdaoutlier<\/em><\/a>) used for this project allowed to independently modify the values of P, Q and the size of the corresponding central regions. Therefore, allowing the sensitivity detection level to be tuned. Outliers detected using the functional boxplot are considered magnitude outliers. In addition, for this blog, anomalies that characterise as both shape and magnitude outliers will also be highlighted, which the standard TVDMSS does not include. This was done by detecting magnitude anomalies while not removing shape anomalies.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-7534cb2 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"7534cb2\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-03a8dbe\" data-id=\"03a8dbe\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-0dcf8f3 elementor-widget elementor-widget-heading\" data-id=\"0dcf8f3\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">2.2 &nbsp;Functional outlyingness map using directional outlyingness&nbsp;<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-bb04e37 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"bb04e37\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-8a93407\" data-id=\"8a93407\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-393e3fb elementor-widget elementor-widget-text-editor\" data-id=\"393e3fb\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>A large number of outlier detection methods rely on using functional depths; as we just saw, this is true for TVDMSS. <a href=\"https:\/\/marcgenton.github.io\/2019.DG.CSDA.pdf\" target=\"_blank\" rel=\"noopener\">Dai &amp; Genton (2019)<\/a> argue that these might fall short of properly describing the centrality of functions and not detect anomalies characterised by their directional outlyingness. Therefore, using the work of <a href=\"https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/10618600.2017.1366912\" target=\"_blank\" rel=\"noopener\">Rousseeuw et al. (2018)<\/a>, <a href=\"https:\/\/marcgenton.github.io\/2019.DG.CSDA.pdf\" target=\"_blank\" rel=\"noopener\">Dai &amp; Genton (2019)<\/a> came up with the functional directional outlyingness which accounts for both the pointwise depth and the directional anomalies.<\/p><p>Outliers are detected using a functional outlier map (FOM). This is visually represented by a scatterplot displaying points and a cut-off line. If the points are out of the cut-off region they are considered outliers. The points are plotted using a functional directional outlyingness value on the x-axis; this reflects the total outlyingness of a given curve and the variation of the directional outlyingness on the y-axis; which represents the shape outlyingness of that curve. To fit this method,\u00a0 we used the function <a href=\"https:\/\/rdrr.io\/cran\/mrfDepth\/man\/fOutl.html\" target=\"_blank\" rel=\"noopener\"><em>fOult <\/em><\/a>from the R package <em><a href=\"https:\/\/cran.r-project.org\/web\/packages\/mrfDepth\/mrfDepth.pdf\" target=\"_blank\" rel=\"noopener\">mrfDepth<\/a>. <\/em>This function does not allow to change the level of anomaly detection, therefore, no tuning is possible for how strict we want to be with our anomalies.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-c5cf148 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"c5cf148\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-40ade29\" data-id=\"40ade29\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-350dc8a elementor-widget elementor-widget-heading\" data-id=\"350dc8a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">3. Results<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-480ba12 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"480ba12\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-005a50f\" data-id=\"005a50f\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-c8d8765 elementor-widget elementor-widget-text-editor\" data-id=\"c8d8765\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>This section presents the results from applying the TVDMSS and FOM(DO) to the data presented in Figure 1.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-7b7de42 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"7b7de42\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-e487b5e\" data-id=\"e487b5e\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-790afb9 elementor-widget elementor-widget-heading\" data-id=\"790afb9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">3.1 Detecting anomalies using Total variation depth and modified shape similarity index<br><\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-f7023a0 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"f7023a0\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-1ec13b6\" data-id=\"1ec13b6\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-fbc4457 elementor-widget elementor-widget-text-editor\" data-id=\"fbc4457\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>After manually tuning the TVDMSS algorithm to a level of detection which empirically seemed reasonable (P = 0.5 and Q = 1.1) we obtain the following results. As shown in Figure 2, three shape anomalies were detected using the classical boxplot of MSS. These are represented by the red circle below the boxplot.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-b78d8a2 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"b78d8a2\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-c7ed0d1\" data-id=\"c7ed0d1\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-3c96b8a elementor-widget elementor-widget-image\" data-id=\"3c96b8a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"768\" height=\"392\" src=\"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu2-768x392.png\" class=\"attachment-medium_large size-medium_large wp-image-1124\" alt=\"Figu2\" srcset=\"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu2-768x392.png 768w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu2-300x153.png 300w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu2-1024x523.png 1024w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu2-1536x784.png 1536w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu2.png 1583w\" sizes=\"(max-width: 768px) 100vw, 768px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-9031a48 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"9031a48\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-bc7b4d8\" data-id=\"bc7b4d8\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-515d21b elementor-widget elementor-widget-spacer\" data-id=\"515d21b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"spacer.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-spacer\">\n\t\t\t<div class=\"elementor-spacer-inner\"><\/div>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-4a1b492 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"4a1b492\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-7b98021\" data-id=\"7b98021\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-5b3775a elementor-widget elementor-widget-text-editor\" data-id=\"5b3775a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Next, the magnitude outliers were identified using the functional boxplot as seen in Figure 3. As we can see, two magnitude anomalies were detected, represented by the blue lines. The pale area corresponds to the 50% central region and the slightly darker lines are the anomaly thresholds.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-d04134f elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"d04134f\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-f085f2f\" data-id=\"f085f2f\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-75f2b4a elementor-widget elementor-widget-image\" data-id=\"75f2b4a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"768\" height=\"502\" src=\"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu3-768x502.png\" class=\"attachment-medium_large size-medium_large wp-image-1125\" alt=\"Figu3\" srcset=\"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu3-768x502.png 768w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu3-300x196.png 300w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu3-1024x670.png 1024w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu3.png 1349w\" sizes=\"(max-width: 768px) 100vw, 768px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-d55720a elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"d55720a\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-22791f4\" data-id=\"22791f4\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-36c8f98 elementor-widget elementor-widget-spacer\" data-id=\"36c8f98\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"spacer.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-spacer\">\n\t\t\t<div class=\"elementor-spacer-inner\"><\/div>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-65f6b5d elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"65f6b5d\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-b04419a\" data-id=\"b04419a\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-25ad5c9 elementor-widget elementor-widget-text-editor\" data-id=\"25ad5c9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Finally, Figure 4 displays all the anomalies detected using the TVDMSS. We can see that one anomaly was classified as both a shape and magnitude anomaly. Furthermore, the shape anomaly detected by the boxplot of MSS corresponding to the lowest red line at t = 0 seems questionable. Empirically, it does not look very different from the other curves.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-9a7b33e elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"9a7b33e\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-ace028e\" data-id=\"ace028e\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-448afec elementor-widget elementor-widget-image\" data-id=\"448afec\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"768\" height=\"416\" src=\"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu4-768x416.png\" class=\"attachment-medium_large size-medium_large wp-image-1126\" alt=\"Figu4\" srcset=\"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu4-768x416.png 768w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu4-300x162.png 300w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu4-1024x554.png 1024w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu4.png 1471w\" sizes=\"(max-width: 768px) 100vw, 768px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-520838c elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"520838c\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-a359576\" data-id=\"a359576\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-e13f18f elementor-widget elementor-widget-spacer\" data-id=\"e13f18f\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"spacer.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-spacer\">\n\t\t\t<div class=\"elementor-spacer-inner\"><\/div>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-68e1363 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"68e1363\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-fd5f76a\" data-id=\"fd5f76a\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-00410ce elementor-widget elementor-widget-heading\" data-id=\"00410ce\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">3.2 Detecting anomalies using functional outlyingness map using directional outlyingness<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-a1b14f7 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"a1b14f7\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-c32e36c\" data-id=\"c32e36c\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-a942bd6 elementor-widget elementor-widget-text-editor\" data-id=\"a942bd6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>The FOM(DO) has not classified any of the curves as outliers. As shown in Figure 5, all the points lie within the cut-off line. As explained in Section 2.2, the function <a href=\"https:\/\/rdrr.io\/cran\/mrfDepth\/man\/fOutl.html\" target=\"_blank\" rel=\"noopener\"><em>fOult<\/em><\/a> used to create this cut-off line does not allow to modify the threshold level. Therefore, to test if the FOM(DO) would detect a more extreme outlier, a curve was added to the data with value 14 for all t. Figure 6 show that the FOM(DO) does detect this more severe outlier, which is represented by the red triangle. This was also tested with a curve at value 13, however, it was not detected as an outlier although it\u2019s shape was very different from any other curve.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-492c58f elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"492c58f\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-4570a3a\" data-id=\"4570a3a\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-1d67a7f elementor-widget elementor-widget-image\" data-id=\"1d67a7f\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"255\" src=\"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu56-1024x255.png\" class=\"attachment-large size-large wp-image-1127\" alt=\"Figu56\" srcset=\"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu56-1024x255.png 1024w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu56-300x75.png 300w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu56-768x191.png 768w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu56-1536x382.png 1536w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-content\/uploads\/sites\/37\/2021\/12\/figu56.png 1842w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-97994a9 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"97994a9\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-c1a9ab6\" data-id=\"c1a9ab6\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-c2cde73 elementor-widget elementor-widget-spacer\" data-id=\"c2cde73\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"spacer.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-spacer\">\n\t\t\t<div class=\"elementor-spacer-inner\"><\/div>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-e08e06e elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"e08e06e\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-ca5296c\" data-id=\"ca5296c\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-1e11022 elementor-widget elementor-widget-heading\" data-id=\"1e11022\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">3.3 Comparing both anomaly detection methods<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-dc4d851 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"dc4d851\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-faf3307\" data-id=\"faf3307\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-b673fdb elementor-widget elementor-widget-text-editor\" data-id=\"b673fdb\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Comparing the results obtained by the two anomaly detection methods is limited by the R functions and packages that were used to produce them. Keeping that in mind, there are four major points on which these two can be compared. First, the TVDMSS algorithm was 10 times fast than running the FOM(DO), (0.05s vs 0.5). Secondly, the TVDMSS performances better when using the default values of the respective functions. Then, the FOM(DO) did not allow for any tuning, whereas this was easy for the TVDMSS. Therefore, once tuned the TVDMSS performed relatively well, by detecting all the true outliers.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-750bdea elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"750bdea\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-eb885e5\" data-id=\"eb885e5\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-933443a elementor-widget elementor-widget-heading\" data-id=\"933443a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">4 Limitations and Conclusion<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-fbecbcc elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"fbecbcc\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-b568f74\" data-id=\"b568f74\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-1b611fa elementor-widget elementor-widget-text-editor\" data-id=\"1b611fa\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Both outlier detection methods have their respective limitations. Because of how the algorithm of the TVDMSS works, it favours shape over magnitude anomalies. Since shape anomalies are first detected then removed. This might be an issue for someone who is interested in anomalies that display both behaviours. This was therefore added as an extension to the TVDMSS, as discussed in Section 2.1. Another significant limitation of the TVDMSS is that it relies heavily on being tuned, the values of P, Q and the percentage size of the central region must be set using trial and error. Finally, the TVDMSS is based on both classical and functional boxplots, which themselves rely on sufficient sample size. For the FOM(DO), we have seen that it was not possible to tune the cut-off line to allow a less strict outlier detection threshold. However, there is also a theoretical limitation to its application in this project. The FOM(DO) performs well with multivariate functional data and although it does work in univariate cases, it does not have much power.<\/p><p>Therefore, we have seen that although both the TVDMSS and the FOM(DO) are two outlier detection methods which have strong theoretical founding, they both have the potential to detect shape and magnitude anomalies effectively. However, their performances is strongly affected by the context in which these are used and how they are implemented. For the data used in this project, the TVDMSS performed a lot better than the FOM(DO), almost perfectly detecting all three true outliers. Although, this is largely due to the inability to tune the FOM(DO). Therefore, it would be interesting to try and implement both these algorithms manually, enabling full control over the tuning of the parameters and thus re-evaluate their respective performances.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>We call an anomaly, an observation that differs significantly from the rest of the data. These are important to detect as they can indicate the existence of a problem. Early identification could even prevent the problem recurring and avoid subsequently worse issues. <\/p>\n","protected":false},"author":40,"featured_media":1051,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"ocean_post_layout":"full-width","ocean_both_sidebars_style":"","ocean_both_sidebars_content_width":0,"ocean_both_sidebars_sidebars_width":0,"ocean_sidebar":"0","ocean_second_sidebar":"0","ocean_disable_margins":"enable","ocean_add_body_class":"","ocean_shortcode_before_top_bar":"","ocean_shortcode_after_top_bar":"","ocean_shortcode_before_header":"","ocean_shortcode_after_header":"","ocean_has_shortcode":"","ocean_shortcode_after_title":"","ocean_shortcode_before_footer_widgets":"","ocean_shortcode_after_footer_widgets":"","ocean_shortcode_before_footer_bottom":"","ocean_shortcode_after_footer_bottom":"","ocean_display_top_bar":"default","ocean_display_header":"default","ocean_header_style":"","ocean_center_header_left_menu":"0","ocean_custom_header_template":"0","ocean_custom_logo":0,"ocean_custom_retina_logo":0,"ocean_custom_logo_max_width":0,"ocean_custom_logo_tablet_max_width":0,"ocean_custom_logo_mobile_max_width":0,"ocean_custom_logo_max_height":0,"ocean_custom_logo_tablet_max_height":0,"ocean_custom_logo_mobile_max_height":0,"ocean_header_custom_menu":"0","ocean_menu_typo_font_family":"0","ocean_menu_typo_font_subset":"","ocean_menu_typo_font_size":0,"ocean_menu_typo_font_size_tablet":0,"ocean_menu_typo_font_size_mobile":0,"ocean_menu_typo_font_size_unit":"px","ocean_menu_typo_font_weight":"","ocean_menu_typo_font_weight_tablet":"","ocean_menu_typo_font_weight_mobile":"","ocean_menu_typo_transform":"","ocean_menu_typo_transform_tablet":"","ocean_menu_typo_transform_mobile":"","ocean_menu_typo_line_height":0,"ocean_menu_typo_line_height_tablet":0,"ocean_menu_typo_line_height_mobile":0,"ocean_menu_typo_line_height_unit":"","ocean_menu_typo_spacing":0,"ocean_menu_typo_spacing_tablet":0,"ocean_menu_typo_spacing_mobile":0,"ocean_menu_typo_spacing_unit":"","ocean_menu_link_color":"","ocean_menu_link_color_hover":"","ocean_menu_link_color_active":"","ocean_menu_link_background":"","ocean_menu_link_hover_background":"","ocean_menu_link_active_background":"","ocean_menu_social_links_bg":"","ocean_menu_social_hover_links_bg":"","ocean_menu_social_links_color":"","ocean_menu_social_hover_links_color":"","ocean_disable_title":"default","ocean_disable_heading":"default","ocean_post_title":"","ocean_post_subheading":"","ocean_post_title_style":"","ocean_post_title_background_color":"","ocean_post_title_background":0,"ocean_post_title_bg_image_position":"","ocean_post_title_bg_image_attachment":"","ocean_post_title_bg_image_repeat":"","ocean_post_title_bg_image_size":"","ocean_post_title_height":0,"ocean_post_title_bg_overlay":0.5,"ocean_post_title_bg_overlay_color":"","ocean_disable_breadcrumbs":"default","ocean_breadcrumbs_color":"","ocean_breadcrumbs_separator_color":"","ocean_breadcrumbs_links_color":"","ocean_breadcrumbs_links_hover_color":"","ocean_display_footer_widgets":"default","ocean_display_footer_bottom":"default","ocean_custom_footer_template":"0","slim_seo":{"title":"Anomaly detection for functional data - Thomas Newman","description":"We call an anomaly, an observation that differs significantly from the rest of the data. These are important to detect as they can indicate the existence of a p"},"ocean_post_oembed":"","ocean_post_self_hosted_media":"","ocean_post_video_embed":"","ocean_link_format":"","ocean_link_format_target":"self","ocean_quote_format":"","ocean_quote_format_link":"post","ocean_gallery_link_images":"off","ocean_gallery_id":[],"footnotes":""},"categories":[11],"tags":[],"class_list":["post-789","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-statistics","entry","has-media"],"_links":{"self":[{"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-json\/wp\/v2\/posts\/789","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-json\/wp\/v2\/users\/40"}],"replies":[{"embeddable":true,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-json\/wp\/v2\/comments?post=789"}],"version-history":[{"count":100,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-json\/wp\/v2\/posts\/789\/revisions"}],"predecessor-version":[{"id":1171,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-json\/wp\/v2\/posts\/789\/revisions\/1171"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-json\/wp\/v2\/media\/1051"}],"wp:attachment":[{"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-json\/wp\/v2\/media?parent=789"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-json\/wp\/v2\/categories?post=789"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/thomas-newman\/wp-json\/wp\/v2\/tags?post=789"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}