| I still like using the 95th percentile number. I wasn’t suggesting we eliminate that. Instead, I was suggesting that the weekend traffic levels are lowering the 95th%ile number. If I understand how 95th percentile is calculated, then If two sevenths of our data is well below average, then it will pull the 95th percentile down and we won’t get a true measurement of utilization during the other five sevenths of the time we really care about. I’m not sure how significant the difference would be, but what I’m suggesting is that we calculate the 95th percentile based on the work/school week and discard the weekend data in our calculations. It’s entirely possible my understanding of how the 95th percentile is calculated is wrong, so let me know if I’m on the wrong track with this.
I’m not against reporting the 98%ile or 99%ile numbers in addition if you think it would be useful. I agree 95th%ile eliminates the short duration spikes, which may or may not be a good thing. I’m not really a statistician, so I can’t really argue for or against it with any conviction.
Ron
On Mar 4, 2016, at 12:44 PM, Jeff Bartig <> wrote:
Personally, I find the
manual report Ron has been doing to be much more useful. It is easy to
see the trouble areas. I closely matches a similar spreadsheet I've
been using at times to track TR/CPS usage.
Ron: I don't think I understand your comment about 95%ile and low usage
on the weekends. I think using some type of %ile algorithm is the
correct approach, since all it is doing is throwing out the top 5% of
measurement intervals and then reporting the highest remaining
interval. The weekends would in some cases be the lowest intervals and
would never factor into the measurement. For 95%ile, I think the more
important question is if it is throwing away important high usage
intervals. %5 of a day is 72 minutes. 5% of a week is 8.4 hours. 5%
of a month is 36 hours. If we are looking at a weekly report, we could
have someone doing an hour long data transfer every day and filling the
circuit full. A 95%ile value might toss out these peaks.
Reporting on the maximum measurement interval over a week often isn't
useful either. Sometimes measurement errors happen. Sometimes it might
have been just 1 minute of high traffic for the entire week. Maybe
also reporting 98 or 99%ile in addition to 95%ile would be a solution
for this.
Jeff
Ron Milford wrote:
Unfortunately, I think including the 52 week avg throws off the trending
numbers if it includes the outlier weeks/months. Did you look at the
spreadsheet I included in my first email? It pretty clearly shows where
the problem circuits are and for how long they have been a problem. With
that type of format on a web-based report, you can easily identify
short term trends visually and use the suggested
quarterly/biannual/annual report(s) for longer term analysis.
Thanks,
Ron
On Mar 4,
2016, at 11:16 AM, Jon-Paul Herron <> wrote:
Thanks this is a good thread, and since
we’re in the process of creating a report library to describe the
purposes, methods, and data model of each report, this will be useful
info for us.
Some other
comments inline: On Mar 4, 2016, at 10:54 AM, Ron Milford <>
wrote:
It was
pointed out to me that the “Estimated # Weeks until 40% is reached” is
used to estimate number of potential augments for budgeting purposes. I
think we can improve the value and accuracy of that number and the “Avg
Growth per week” numbers by calculating it based on the most standard
weeks of utilization.
One possibility would be to eliminate the summer months, winter
holiday weeks and other outlier weeks like the week of SC when
calculating the trend. Alternatively or in conjunction a direct
comparison of the same timeframes in the previous year or semester will
give us a longer term, but more accurate growth curve.
Since the occurrence of
outlier weeks varies from year to year, this would most likely have to
be an ad-hoc report done on an annual, biannual or quarterly basis with
the ability to specify the outlier weeks to exclude.
If anyone has any other ideas of how to
improve the accuracy of the data please let me know.
I can see how the suggestions would be more accurate, but is
that additional accuracy going to make a difference in the
decision-making for when and how urgently to augment? Looking at the
current report, I can see, for instance that 55944 is at the top of the
list, and yet it’s 55918 that’s really been busy for a very long time.
It would seem to me that the combination of 52 week average and this
trend would help with this.
I might also suggest moving to a web-based report with emailed
link. Thanks, Ron On Mar 4, 2016,
at 10:09 AM, Ron Milford <>
wrote:
I’m not
sure who else uses the AL2S Weekly Bandwidth Capacity Report, but I
would like to propose a few changes.
I
find that some of the data is very unreliable or not useful due
primarily to the major drops in traffic during summer months and school
holidays. I don’t know who uses that data if anyone, but due to the
amount of data the report has to sift through it is taking several hours
to run. I would suggest we eliminate the following portions of the
report since they are misleading at best.
“Estimated # Weeks until 40% is reached” “Avg
Growth per week (Mbps)”
These may have some use to someone, but I’m not sure it makes
sense in the context of this report and I expect eliminating them from
this report would speed things up. "Inbound % Last
52 week(s)” "Outbound % Last 52 week(s)"
I also would like to see the
output in a slightly different format if possible. Every week, I update
the attached spreadsheet which is much more informative. It would be
great to get an html report in this type of format showing the actual
numbers over time. That way it is easier to compare to in school vs out
of school and see trends relatively quickly and easily.
Additionally we might want to
consider revising the reporting period to Monday 0:00 - Friday 24:00
(timezone TBD). I fear we are under-reporting our true bandwidth demand
by factoring in the lower utilization weekends in our 95% calculations
(unless someone has already taken this into account).
Thanks, Ron On Mar 4, 2016, at 9:26 AM,
wrote:
AL2S Weekly Bandwidth Capacity Report for
Fri Mar 4 12:30:02 UTC 2016 This report is averaging and trending
traffic based on the past 52 weeks worth of available data for each
circuit. In/out bound utilization based on last 1 week(s) of data.AL2S 95th Percentile Circuit
Utilization and Trending. A value of N/A means that a positive slope
was not able to be calculated, either because the overall trend was
downwards or if there was no data.A circuit's name in red means that it is exceeding 30%
traffic in either direction for the last 1 week(s).An estimate in red means that the prediction is
within 3 months of exceeding the threshold.
-- Jeff
Bartig
Interconnection Architect
Internet2 AS11164 / AS11537
+1- 608-616-9908
|
