Skip to Content.
Sympa Menu

perfsonar-user - Re: [perfsonar-user] Infiniband measurements?

Subject: perfSONAR User Q&A and Other Discussion

List archive

Re: [perfsonar-user] Infiniband measurements?


Chronological Thread 
    < Chronological >      < Thread >
  • From: "David J. Chaffin" <>
  • To: Mark Feit <>, "" <>, Otto J Wittner <>
  • Subject: Re: [perfsonar-user] Infiniband measurements?
  • Date: Wed, 2 Nov 2022 22:04:18 +0000
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uark.edu; dmarc=pass action=none header.from=uark.edu; dkim=pass header.d=uark.edu; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Hm1h79wjssJ3zSBEpjpLkFHOTW/0xNOfvn3fr8a1jf8=; b=mlEFzBB+6SfKIeCEl+j9hsArDWbZBmO8i1s4+JudlLPAGbQQ2w839u9hV2iX3ab+sF9tKyGOuF0xWDGYFFbXvTqQmAXNH3kKe6rj1yqvdjFzASLvI1UMedU4vWgl39VNltdmiYLAKIE8jT6jF6Oq0AiIoL9oH6vDB0UPUYQhFsjgWCeX7GN6njt7bGaof+x4I3RyixibVXGTEZrvqUyH9ytBxCK9h8KudxPpGH3IZ+BM5+f2Wd/uuKUfVGaXCr+haWlUszE70YdvzurqrL400vMpHOmwEJui1QVwTdpDLxNCQb53I6E5wFJzudIMB02WEPY5G797m/xzqJpbFZCzYQ==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eGzR8SPWoG0iCG0zIxIgnI6oYeHAubJVKUs+7i2NNXIUif99vpsIOB+W6D4r/YuCSDQ7I9uGUR/MCjKbytDtUc6XADaDzyWZ5aC/8Mn9lFhEm4Rptyf1KuxR9wYhWjw8qF/NBZK19dw3Tw0qE/x+AxggWv8uR80O0c5wAy96Rdaz6rUeA/wrGUv5t6RlMtA9bvZyULj49l7Pc0s3EZuby1QbT8eaQiSKL+HAR2FTpGBY6TKFAhqbwYgkNqX9WYrdaSSfDqDj4aio5C2tJR/UKg4tjaPGlx5PAXWX8koxIdnYhcJbZSFNtTTvo5uFNqXGEeR1wN+6fs+GegPbjIc2jw==
  • Msip_labels:
Perfsonar should work over ipoib interfaces but there is little reason to do it.  You'd also need a perfsonar server on every cluster node. You don't have to search for slow routers and firewalls. The very useful program iblinkinfo will tell you every link that's not up to par, usually because of cables, and ib_send|read_bw|lat will test the links.  They're usually 90-some % of advertised speed.

--
David
From: <> on behalf of Otto J Wittner <>
Sent: Wednesday, November 2, 2022 5:02 AM
To: Mark Feit <>; <>
Subject: Re: [perfsonar-user] Infiniband measurements?
 


On 31.10.2022 17:13, Mark Feit wrote:
> Otto J Wittner writes:
>
> Has anyone applied perfSONAR in an Infiniband topology and performed
> measurements? If yes, are there any opticals worth being aware of?
>
> I haven’t personally, but as long as what perfSONAR sees looks like IP,
> there should be no problem with it

Yes, right. But to what extent is something interesting/relevant
measured then I wonder...? There seem to be both an IP stack and a
couple of other stacks in Nvidia's Infiniband architecture:
https://nam11.safelinks.protection.outlook.com/?url="https%3A%2F%2Fdocs.nvidia.com%2Fnetworking%2Fdisplay%2FMLNXOFEDv494170%2FIntroduction&amp;data=05%7C01%7Cdchaffin%40uark.edu%7C7045b31327d148a40db008dabcba023a%7C79c742c4e61c4fa5be89a3cb566a80d1%7C0%7C0%7C638029804465723887%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=0muKslHKVx6SJ%2BHhuG8nadDdXe3YLJVD6d%2B1gpNi9N8%3D&amp;reserved=0
I suspect the uDAPL API is potentially more relevant than the IP API...

I was just "fishing" for people how might have look into this.

(Mark / Andy, you mentioned a name in this context earlier which I noted
down... but my notes seem to have "hidden themselves")

>
> --Mark
>

--
Otto J Wittner  tel +47 99550566
Sikt            https://nam11.safelinks.protection.outlook.com/?url="http%3A%2F%2Fsikt.no%2F&amp;data=05%7C01%7Cdchaffin%40uark.edu%7C7045b31327d148a40db008dabcba023a%7C79c742c4e61c4fa5be89a3cb566a80d1%7C0%7C0%7C638029804465723887%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=57oh4DveuGuDqg%2Fbpw2uYraAVKtbCMXyjjxpNDGoyK4%3D&amp;reserved=0

Archive powered by MHonArc 2.6.24.

Top of Page