perfSONAR User Q&A and Other Discussion

["Dale W. Carder" <>]

You don't necessarily need to use PTP on your endpoints, as you could 
probably discipline them individually to GPS.  I was thinking of
experimenting along these lines with some boxes we have w/ rubidium
oscillators, but I can't seem to get a correct build environment offhand 
to get owamp on there (granted, I spent like 3 minutes).

That said, I'm not too sure what I am going to see in a LAN environment
other than what I expect, tiny buffers periodically filling up and 
middleboxes that do not improve performance.  So, I am pretty curious
about the use-case (regardless of PerfSONAR) because I would like to 
know what I'm missing here.

Dale


Thus spake Magorian, Daniel F. 
()
 on Fri, Feb 19, 2016 at 09:07:25PM +0000:
> Ok, Jason wrote me back privately saying similar things, and so I agree 
> that PTP and LAN use is out-of-scope for PerfSONAR.  
> 
> So I'll stop hammering on PerfSONAR to become a LAN tool that isn't in 
> scope for the project.  And in return, PerfSONAR folks should stop 
> pretending it can do stuff it can't, and firmly define their WAN scope to 
> not lead people new to measurement astray by implying that it can be used 
> for LAN measurement.  Someone should dump that misleading PTP statement and 
> just define the scope properly.  I suggest this rewrite put somewhere 
> prominent:
> 
> "PerfSONAR's primary mission is identifying network problems along paths 
> between WAN domains.  The latency in those paths tends to be large enough 
> that NTP's millisecond accuracy is sufficient for most testing.  
> Intra-campus LAN path measurements needing sub millisecond timing should 
> use other tools than PerfSONAR that use PTP and are better suited to those 
> tasks.   At some point the Performance Working Group should form a task 
> area to evaluate and recommend the use of appropriate LAN latency 
> measurement tools, such as smart SFP-based systems"
> 
> > Because I think that pretty much nobody is on the same page as you are, 
> > but coming from you, it intrigues us...   What I would be very interested 
> > in are: "An Exposition Of The Joys Of Short Path Latency Analysis"  and  
> > "How To Implement PTP Timing In An Academic Enterprise Network for OWAMP 
> > Discipline"
> 
> Thanks. Well, I know Andrew Gallo of GWU gets where I'm coming from because 
> he was also asking for PTP, but anyway...     I think the main difference 
> is that I've been working with new smart SFPs and other people have not.  
> So they may be unaware of how purpose-built cheap hardware has changed the 
> whole measurement landscape compared to more traditional software-based 
> measurement tools.  Call it part of the Internet of Things:  previously 
> dumb stuff like SFPs now acquiring smarts.
> 
> Yeah, I could write some white papers and how-tos on it, but you know those 
> don't usually get read or paid attention to.
> 
> What might be the right approach is a new LAN measurement task area within 
> the Performance Working Group, given that PerfSONAR is the endorsed task 
> area for WAN measurement.  Then those white papers could be the new LAN 
> task area charter, and folks like me who are more interested in the LAN 
> side could sign off this list and leave the WAN PerfSONAR fans in peace.  
> That task area could then focus on smart SFPs and other new tech for LAN 
> measurement, and obviously people could tune into both if they measure 
> both.   I think it nicely fits the I2 mission to reinforce both LAN and WAN 
> sides these days and not just be all about WANs.
> 
> The problem is, I have no idea how to do this.   I need help from someone 
> with an administrative bent to navigate that process...
> 
> Dan
> 
> -----Original Message-----
> From: 
> 
>  
> [mailto:]
>  On Behalf Of Alan Whinery
> Sent: Friday, February 19, 2016 2:58 PM
> To: 
> 
> Subject: Re: [perfsonar-user] feedback on PTP
> 
> Dan,
> 
> You can't have both:
> >> Dan in his talk today said  something like  "Well, anything below 1
> >> mS is basically instantaneous" .  Again, spoken with a strong WAN
> >> bias and little understanding of intra-campus paths.
> 
> and
> 
> > Sorry, without the ability to see below 1 mS, there is no such thing
> >> as "a general sense of campus network performance".
> 
> Sorry, but I get value value from analyzing packet loss exclusively,
> which probably points out the same queuing-related problems that I
> assume you're looking for with latency.
> 
> And assailing the perfSONAR project's foci is a little like writing your
> congressman, it has ceremonial flair, little value otherwise.
> 
> What I would be very interested in are:
> 
>       "An Exposition Of The Joys Of Short Path Latency Analysis"
>       and
>       "How To Implement PTP Timing In An Academic Enterprise Network  
>       For OWAMP Discipline"
> 
> or existing links.
> 
> Because I think that pretty much nobody is on the same page as you are,
> but coming from you, it intrigues us, because you say smart stuff, and
> there's much implicit "exercise for the reader" in your in-band comments
> that I've seen.
> 
> -Alan
> 
> 
> On 2/19/2016 8:40 AM, Magorian, Daniel F. wrote:
> > Actually, surprise!  Smart SFP TWAMP reflectors (that also can sniff
> > packets for remote pcap and other useful things) are $300/ea, only
> > about 2x what the small nodes cost, and the central initiator that
> > times the TWAMP reflectors is also cheap.  As an SFP it's also super
> > easy to replace ordinary SFPs with.  I bought the two dozen and the
> > traffic initiator for < $5k, chump change.
> > 
> >> small-footprint nodes can be deployed all over campus, to give us a
> >> general sense of network performance
> > 
> > Sorry, without the ability to see below 1 mS, there is no such thing
> > as "a general sense of campus network performance".  Either you can
> > resolve your campus path latency, or you just have lines along the
> > bottom of your graphs at 1 mS, pretty worthless.
> > 
> > Why settle for low resolution when you can have high for close to the
> > same cost?
> > 
> > Dan
> > -----Original Message-----
> > From: Peter Gutierrez 
> > [mailto:]
> >  
> > Sent: Friday, February 19, 2016 1:13 PM
> > To: Magorian, Daniel F.; 
> > ;
> >  
> > 
> > Cc: Andrew Gallo; 
> > ;
> >  Jason Zurawski
> > Subject: Re: [perfsonar-user] feedback on PTP
> > 
> > On 02/19/2016 12:43 PM, Magorian, Daniel F. wrote:
> >> Dan and Jason, this discussion on PTP pretty much misses the point
> >> entirely,
> > 
> > 
> > In my opinion, accurate intra-campus latency measurement has value, but I
> > don't expect it from a cigarette-box server any more than I expect them to
> > provide 100Gbps interfaces for bandwidth testing. As long as these
> > low-cost, low-power, small-footprint nodes can be deployed all over
> > campus, to give us a general sense of network performance, it's a yoooge 
> > win
> > .
> > 
> > Deploying expensive, pizza-box servers with appropriate hardware to do
> > more advanced latency/bandwidth measurements in the places I need it is a
> > worthwhile investment as well.
> > 
> > PeterG
> > 
> > 
> >> https://www.perfsonar.net/deploy/timekeeping-with-ntp/
> > 
> >> and demonstrates how software guys rarely see the need for/advantages
> >> of dedicated hardware.  Follow that logic to its conclusion and you can
> >> do away with small nodes altogether, and just create a centralized VM
> >> box with virtual measurement node instances cabled to wherever.  Or
> >> containers these days, which at least are lighter weight.
> > 
> >> Obviously no one thinks that *synchronizing* PTP is cost-effective yet,
> >> as would be needed for OWAMP.
> > 
> >> The point you're missing is that TWAMP, while it does traverse the path
> >> twice, goes and comes back to the same PTP initiator.  Therefore it has
> >> no need to sync to anything else, and is accurate internally to ~1 uS
> >> so intra-campus paths can be measured.
> > 
> >> Since you're showing no interest in measuring such intra-campus paths,
> >> you need to stop saying "perfsonar world domination" and change it to
> >> "perfsonar inter-campus WAN world domination".  And if that's your
> >> mission and the scope of the project, software NTP-based timing to mS
> >> works fine.
> > 
> >> Just know that you're making the project unusable for an enormous new
> >> intra-campus market, due to WAN-oriented bias.
> > 
> >> Dan in his talk today said  something like  "Well, anything below 1 mS
> >> is basically instantaneous" .  Again, spoken with a strong WAN bias and
> >> little understanding of intra-campus paths.  If your north-south
> >> core-to-edge campus traffic is talking 1000 uS, you have serious
> >> performance issues that really need to be looked into and probably have
> >> users beating you up daily.  Which normally everyone blames on
> >> firewalls and repeats that tired litancy without bothering to actually
> >> measure, because they can't without the right tools.
> > 
> >> In reality good Cisco or Palo Alto campus firewalls introduce ~ 50 -
> >> 100 uS latency when working properly, as does your IPS.  I know,
> >> because I measure ours and watch for latency spikes on the cacti
> >> graphs.
> > 
> >> My overall point is, you could extend Perfsonar's usability to
> >> intra-campus path measurement relatively easily with some architectural
> >> changes, but nobody wants to because of WAN bias.
> > 
> >> Sigh.
> > 
> >> Dan Magorian
> > 
> > 
> > 
> > 
>