I’m excited to announce we’ve just released an updated version of Xmrit, with powerful new features to easily handle Trended and Seasonal data — no more tedious spreadsheet work required!
To help you get started quickly, I’ve created a short video demonstrating these new features, along with practical tips and tricks. If you want to get a deeper understanding of how these new features work this article includes detailed instructions on Trends and Seasonality, taken from the Xmrit manual.
Finally I want to thank Sean for all his hard work in building these features, alongside a full rewrite of Xmrit using the Apache ECharts library.
Sam, Cedric, and the rest of the Xmrit team
Trends
What is Trended Data?
A trend refers to a sustained movement in the data, either upward or downward. An example of data with a downward trend is shown below.
Why Trended Data is an Issue for XmR Charts
Processes with significant trends often show up as having exceptional variation on XmR charts. In some cases identifying trends as exceptional is desired, as your process normally has no trend and you want to know when a trend appears. However, in some cases your data has a predictable trend, and the exceptional behaviour you want to detect is if there was a change in the trend.
In the trended data example shown before the start and end of the data is identified as exceptional. However it may be that the trend is expected, and the XmR chart is wrongly identifying the exceptional variation.
Xmrit’s Approach to Removing Trends - Linear Regression
To manage trended data Xmrit allows you to “trend your limits” alongside your data. This is done by building a linear regression of your data, and making the regression line the new process average centre line. The natural process limits are then trended around the central regression line, at the same separation as with the untrended original XmR chart.
The trended limits are solid, as they behave like locked limits. The three rules to detect exceptional variation will be calculated using your new trended limits. Below is the example trended data, alongside the trended limits generated by Xmrit. You can see the graph at this link.
How to Generate Trended Limits Using Xmrit
To generate trended limits from your data use the Trends button, circled in red in the image below.
The Trends button works the same way as the Lock Limits and Seasonality buttons, allowing you to select the data the linear regression is calculated from. The popup data table will be automatically populated with all the data already in Xmrit.
Inside the popup you will also see an “Average Line Equation”. This is the formula for the linear regression line that will become your new process average centre line. The first value is the slope of the regression line, and the second value is the intercept. If you want more information on linear regressions we recommend this Wikipedia page.
Choosing What Data to Include in Your Regression Calculation
A key decision you must make when building your regression is which data to include. If your regression includes data you think is exceptional that can distort your limits, making them less sensitive to exceptional variation.
In the example below we have data with an obvious upward trend, but the last ~9 data points appear to have flattened off. You can visually see this by the fact that the last 9 data points are not being flagged as having exceptional variation.
When you trend your limits you have to choose, do you include all data points, even the last 9, or do you only include the beginning part of the trend? This is an important decision, as including all the data points in the regression means no exceptional variation is detected, but including only the first 20 points identifies all of the last 9 data points as exceptional variation.
Can Xmrit Do Non Linear Regressions as the Centre Line?
No, Xmrit can only do simple linear regressions.
Seasonality
What is Seasonal Data?
Data is said to have a seasonality when there is a pattern in the data that recurs at the same point every time the data series repeats. The classic example of a seasonal pattern is temperature over the year; cold Winters, hot Summers, with Spring and Autumn somewhere in between.
But seasonal patterns can occur in lots of different ways in your data:
- Shopping malls see more visitors on the weekend compared to weekdays
- Tax-related services see a spike in usage near tax filing deadlines
- Gaming activity often surges during school holidays
- Business to business sales slow down in late December and pick up again mid-January.
Why Seasonal Data is an Issue for XmR Charts
Processes with significant seasonality can cause XmR charts to either miss exceptional variation, or wrongly detect seasonal variation as exceptional.
In the X chart below there are two years and three months of data. The data is highly seasonal, with January always being the lowest month, and December being the highest. This seasonal variation has overwhelmed the XmR chart, creating very wide process limits, making it far less sensitive to exceptional variation than is desired.
This lack of sensitivity is important as there is exceptional variation present on the chart! January, February, and March of year three are all significantly higher values than the same months in years one and two.
Xmrit’s Approach to Removing Seasonality - Seasonal Factors
To allow us to identify exceptional and routine variation on an XmR chart we first need to remove the seasonality from the data. In the literature there are many ways to remove seasonality from your dataset, but for Xmrit’s purposes we have taken the seasonal factor method from Dr. Donald Wheeler’s book “Making Sense of Data: SPC for the Service Sector”.
The seasonal factor method aims to estimate how much of the variation present in your data is explained by seasonal patterns, and then remove it. It does this by comparing the overall average value of your data to the average value of your seasons.
For example if you were analysing temperature data for multiple years, and you wanted to remove the seasonal variation in temperature. If July was on average 20% hotter than the average temperature across your entire dataset the seasonal factor process would reduce every July value by 20%. Conversely if December was on average 12% colder than the average then every December value would be increased by 12%.
Worked Example of Removing Seasonality Using Seasonal Factors
To help explain the seasonal factor process Xmrit has made a spreadsheet of the deseasonalisation process for the two year three month seasonal data that was used as an example previously. Below is a step by step walkthrough of how a single deseasonalised data point was calculated, followed by a comparison before and after of the entire dataset being deseasonalised.
Example Deseasonalisation of a Single Datapoint - January 2020
Calculate the Grand Average Calculate the grand average for the three years of data. In this example the grand average of data is 389.1
- Spreadsheet cell D3 in the annual tab
Calculate the Monthly Average Calculate the average for each month e.g. the average for the three January’s is 245.3.
- Spreadsheet cell E3 in the annual tab
Calculate the Seasonal Factor Calculate the seasonal factor for each month by dividing the monthly average by the overall average e.g. January’s seasonal factor = (245.3/389.1) = 0.6305
- Spreadsheet cell F3 in the annual tab
Calculate the Reciprocal Seasonal Factor For each seasonal factor calculate the reciprocal (1/seasonal factor) e.g. January = 1/0.6305 = 1.586
- Calculated within the formula within spreadsheet cell G3
Calculate the Deseasonalised Data Multiply each original value by its seasonal factor reciprocal e.g. the deseasonalised value for January 2020 = original value x 1.59 = 242.5 x 1.5986 = 384.6
- Spreadsheet cell G3 in the annual tab
Example Deseasonalisation for All Three Years of Example Data
The graphs below show the dataset before and after annual deseasonalising with seasonal factors. You will notice that the year three values that we previously noted as showing exceptional variation are now identified as exceptional on the deseasonalised XmR graph. You can click here to see the deseasonalised data within the Xmrit tool.
How to Remove Seasonality Using Xmrit
To remove the seasonality from the data use the Seasonality button, circled in red in the image below.
The Seasonality button works the same way as the Lock Limits and Trend buttons, allowing you to select the data the seasonal factors are calculated from. The popup data table will be automatically populated with all the data already in Xmrit.
You will notice that the popup data table is coloured, unlike the lock limits and trend popups. The colours correspond to the seasonal factor that will be used to deseasonalise your data. You can double check which seasonal factor will be used to deseasonalise a data point by checking the Season column, and cross referencing with the Seasonal Factor table. In the example popup image January 2020 has a Season value of 1, and the corresponding seasonal factor for season 1 is 0.6304…
Data that has been deseasonalised will appear with the word “Deseasonalised” before the X Plot and MR Plot titles
What Types of Seasonality Can Xmrit Remove
Xmrit can remove four different types of seasonality periods
- Annual Patterns that repeat over a year e.g. temperature of a city where the summer is hot and the winter is cold
- Quarter Patterns that repeat over a quarter e.g. sales numbers which rise at the end of a quarter
- Month Patterns that repeat over a month e.g. factory outputs sometimes spike at the end of a month to hit quotas
- Week Patterns that repeat over a week e.g. the number of visitors to a shopping mall
How to calculate all four types of deseasonalisation is shown in the four tabs in this example spreadsheet.
To select the type of seasonality period you want to remove from your data you can select the period dropdown in the seasonality popup.
Common Error - Why Are All My Values The Same After Deseasonalisation?
If you try to deseasonalise data using a period that is equal to or shorter than your entire dataset it will lead to your deseasonalised having all of your values being the same. In these situations Xmrit will show a warning.
For example, if you ignore this error message, and deseasonalise two months of data annually, all the data points will become the same value.
Can Xmrit remove multiple types of Seasonality?
No, Xmrit can only remove one type of seasonality at a time.