| Jeff,
Ok, so what your saying is, it’s not necessary to eliminate the weekends if we also report on the 98th and 99th percentiles. That seem’s reasonable, so what if we reported on 95%ile, 96.5%ile and either 98 or 99%ile?
For the report format, I think I’d rather have a separate spreadsheet or spreadsheet tab for each percentage measurement. That way it doesn’t break up the the visual continuity and it will be easier to see the trends.
Ron On Mar 4, 2016, at 2:18 PM, Jeff Bartig <> wrote:
Ron,Looking specifically at 95%ile.Full 7 day week: 5% of 7 days is 8.4 hours of the highest usage that gets discarded, and the result of the 95%ile algorithm will be the remaining maximum sample period.Toss out the weekends: 5% of 5 days is 6 hours of the highest usage that gets discarded before selecting the maximum.6 hours represents 3.5% of a full 7 day week.Your proposal to not include the weekends would be similar to just using 96.5%ile.If I was generating a weekly report of interface usage like this, I would follow your spreadsheet example and add a couple of columns:| Circuit | 99%ile | 98%ile | 95%ile | 3/4/16 | 2/26/16 | 2/19/16 | | In | Out | In | Out | In | Out | In | Out | In | Out | In | Out |
I would use 95%ile for the historical data, but add 99%ile and 98%ile for the current week to give a heads up that there might be brief bursty traffic hiding from the 95%ile.JeffRon Milford wrote:I still like using the 95th percentile number. I wasn’t suggesting we eliminate that. Instead, I was suggesting that the weekend traffic levels are lowering the 95th%ile number. If I understand how 95th percentile is calculated, then If two sevenths of our data is well below average, then it will pull the 95th percentile down and we won’t get a true measurement of utilization during the other five sevenths of the time we really care about. I’m not sure how significant the difference would be, but what I’m suggesting is that we calculate the 95th percentile based on the work/school week and discard the weekend data in our calculations. It’s entirely possible my understanding of how the 95th percentile is calculated is wrong, so let me know if I’m on the wrong track with this.
I’m not against reporting the 98%ile or 99%ile numbers in addition if you think it would be useful. I agree 95th%ile eliminates the short duration spikes, which may or may not be a good thing. I’m not really a statistician, so I can’t really argue for or against it with any conviction.
Ron
On Mar 4, 2016, at 12:44 PM, Jeff Bartig <> wrote:
Personally, I find the manual report Ron has been doing to be much more useful. It is easy to see the trouble areas. I closely matches a similar spreadsheet I've been using at times to track TR/CPS usage. Ron: I don't think I understand your comment about 95%ile and low usage on the weekends. I think using some type of %ile algorithm is the correct approach, since all it is doing is throwing out the top 5% of measurement intervals and then reporting the highest remaining interval. The weekends would in some cases be the lowest intervals and would never factor into the measurement. For 95%ile, I think the more important question is if it is throwing away important high usage intervals. %5 of a day is 72 minutes. 5% of a week is 8.4 hours. 5% of a month is 36 hours. If we are looking at a weekly report, we could have someone doing an hour long data transfer every day and filling the circuit full. A 95%ile value might toss out these peaks. Reporting on the maximum measurement interval over a week often isn't useful either. Sometimes measurement errors happen. Sometimes it might have been just 1 minute of high traffic for the entire week. Maybe also reporting 98 or 99%ile in addition to 95%ile would be a solution for this. Jeff Ron Milford wrote:Unfortunately, I think including the 52 week avg throws off the trending numbers if it includes the outlier weeks/months. Did you look at the spreadsheet I included in my first email? It pretty clearly shows where the problem circuits are and for how long they have been a problem. With that type of format on a web-based report, you can easily identify short term trends visually and use the suggested quarterly/biannual/annual report(s) for longer term analysis.
Thanks, Ron
On Mar 4, 2016, at 11:16 AM, Jon-Paul Herron <> wrote:
Thanks this is a good thread, and since we’re in the process of creating a report library to describe the purposes, methods, and data model of each report, this will be useful info for us.
Some other comments inline: On Mar 4, 2016, at 10:54 AM, Ron Milford <> wrote:
It was pointed out to me that the “Estimated # Weeks until 40% is reached” is used to estimate number of potential augments for budgeting purposes. I think we can improve the value and accuracy of that number and the “Avg Growth per week” numbers by calculating it based on the most standard weeks of utilization.
One possibility would be to eliminate the summer months, winter holiday weeks and other outlier weeks like the week of SC when calculating the trend. Alternatively or in conjunction a direct comparison of the same timeframes in the previous year or semester will give us a longer term, but more accurate growth curve.
Since the occurrence of outlier weeks varies from year to year, this would most likely have to be an ad-hoc report done on an annual, biannual or quarterly basis with the ability to specify the outlier weeks to exclude.
If anyone has any other ideas of how to improve the accuracy of the data please let me know.
I can see how the suggestions would be more accurate, but is that additional accuracy going to make a difference in the decision-making for when and how urgently to augment? Looking at the current report, I can see, for instance that 55944 is at the top of the list, and yet it’s 55918 that’s really been busy for a very long time. It would seem to me that the combination of 52 week average and this trend would help with this.
I might also suggest moving to a web-based report with emailed link. Thanks, Ron On Mar 4, 2016, at 10:09 AM, Ron Milford <> wrote:
I’m not sure who else uses the AL2S Weekly Bandwidth Capacity Report, but I would like to propose a few changes.
I find that some of the data is very unreliable or not useful due primarily to the major drops in traffic during summer months and school holidays. I don’t know who uses that data if anyone, but due to the amount of data the report has to sift through it is taking several hours to run. I would suggest we eliminate the following portions of the report since they are misleading at best.
“Estimated # Weeks until 40% is reached” “Avg Growth per week (Mbps)”
These may have some use to someone, but I’m not sure it makes sense in the context of this report and I expect eliminating them from this report would speed things up. "Inbound % Last 52 week(s)” "Outbound % Last 52 week(s)"
I also would like to see the output in a slightly different format if possible. Every week, I update the attached spreadsheet which is much more informative. It would be great to get an html report in this type of format showing the actual numbers over time. That way it is easier to compare to in school vs out of school and see trends relatively quickly and easily.
Additionally we might want to consider revising the reporting period to Monday 0:00 - Friday 24:00 (timezone TBD). I fear we are under-reporting our true bandwidth demand by factoring in the lower utilization weekends in our 95% calculations (unless someone has already taken this into account).
Thanks, Ron On Mar 4, 2016, at 9:26 AM, wrote:
AL2S Weekly Bandwidth Capacity Report for Fri Mar 4 12:30:02 UTC 2016 This report is averaging and trending traffic based on the past 52 weeks worth of available data for each circuit. In/out bound utilization based on last 1 week(s) of data.AL2S 95th Percentile Circuit Utilization and Trending. A value of N/A means that a positive slope was not able to be calculated, either because the overall trend was downwards or if there was no data.A circuit's name in red means that it is exceeding 30% traffic in either direction for the last 1 week(s).An estimate in red means that the prediction is within 3 months of exceeding the threshold.
-- Jeff BartigInterconnection Architect Internet2 AS11164 / AS11537 +1- 608-616-9908
-- Jeff BartigInterconnection Architect Internet2 AS11164 / AS11537 +1- 608-616-9908
|
