perfsonar-user - RE: [perfsonar-user] Dual Interface system
Subject: perfSONAR User Q&A and Other Discussion
List archive
- From: "Garnizov, Ivan (RRZE)" <>
- To: Casey Russell <>, "" <>
- Subject: RE: [perfsonar-user] Dual Interface system
- Date: Tue, 14 Apr 2015 14:40:14 +0000
- Accept-language: en-GB, de-DE, en-US
|
Hi Casey, all, I would like to bring up this topic again as there are some unclear to me configuration requirements, which I can’t seem to find in the documentation.
Now currently I have physical server deployments with 3.4.1, which operate flawlessly with IPv4 and source routing regardless, if I define the default GW in
the /etc/sysconfig/network. BUT IPv6 source routing does not work at all and there is no indication on the routing table (ip -6 rule list command). Which leaves me with fully functional
dual interface system on IPv4 and on IPv6 with only default interface. I have played all the possible options (I could think of) and have the network reinitialized with “service network restart” only. IPv6 did not get registered
and IPv6 source routing does not work there for me. I decide to test the pS 3.4.2 on a VM system, which I can easily get offline access to. The system I installed just today. Now there is progress, but
also a drawback. I have configured a
secondary interface with private IPv4 and IPv6 (real some imaginary) the interface is attached to nowhere.
My
primary interface is also configured and is in my local subnet (not connecting through a gateway). Internet connectivity is present Now when I configure the source routing using the command “mod_interface_route”
for the secondary interface, I can see the route appear in the routing table (ip rule list) for IPv4 and IPv6. BUT when I run the same command for my primary interface the system suddenly is locked for access from outside (I have tested with and without FW). I can operate
through offline management interface only. I get the proper entries in the routing table, but that seems not to be enough for proper communication, even on the same local subnet. Now my question are: 1)
Has the source routing management been changed between pS 3.4.1 and pS 3.4.2? The release notes do not indicate it, but as Casey stated it for 3.4.1
and I have experienced it myself, the IPv6 routing table does not show indication that it is changed in 3.4.1, where in 3.4.2 changes are registered. 2)
What is the recommendation for the interface configuration for a multi-home system in terms of the following config files:
a.
/etc/sysconfig/network-scripts/ifcfg-*
b.
/etc/sysconfig/network 3)
If Casey is correct for the configuration of the system, then how would the sysadmin be able to define the default interface for outgoing requests?
I know that the OS will choose one (I believe it is the first interface based on the name), but that is a very limiting factor for me. 4)
In fact I tried to reproduce the solution of Casey, but for IPv6 only, by removing the statement for a IPv6 GW in /etc/sysconfig/network, but keeping
the one for IPv4 (I am sitting on this branch). I also removed the statements for IPv6GW in the ifcfg-* files and still, IPv6 source routing did not work for me. The only thing I avoided doing is reboot of the server for various reasons. Please help with this case. I would prefer to be able to resolve the source routing case with pS 3.4.1, since that will allow me to work on the other services
awaiting transitioning to newer versions. Best regards, Ivan From: [mailto:]
On Behalf Of Casey Russell Ivan (and group), I think I've resolved the problem myself. It came down to the process I used for configuring my interfaces in the first place. I had initially Put the base interface information (IP,SM Mask, DNS) for
IPV4 and IPv6 in the interface configuration files at /etc/sysconfig/network-scripts/ifcfg-<name> Then I'd placed the Gateway information for the interfaces either in the /etc/sysconfig/network file, or in the routing table with a "route add" statement. You can probably see from the methodology, I'm
used to dealing with primarily single interface systems. Once I deleted the Gateway information (IPv4 and IPv6) from the /etc/sysconfig/network file, and inserted it into the individual interface files at /etc/sysconfig/network-scripts/ifcfg-<name>. I rebooted
the system, ran the mod_interface_route script and things appear to be working now from a traffic standpoint.
I still don't see quite what I thought I'd see from the "ip -6 rule list" command, but traffic flows in and out correctly via both interfaces now (IPv4 and IPv6). Thank you for your helpful prodding in the right direction.
Casey Russell Network Engineer Kansas Research and Education Network 2029 Becker Drive, Suite
282 Lawrence, KS 66047 (785)856-9820 ext 9809 On Mon, Jan 26, 2015 at 9:10 AM, Casey Russell <> wrote: Ivan, >>
The “magic” of /opt/perfsonar_ps/toolkit/scripts/mod_interface_route is not purposed to fix routing. The idea is to make server replies exit through the interface that has received inquiries. Sure, I never expected the script to have any effect on routing beyond right here at the host (Linux policy routing). Just as
you described, I expect the script to help the host determine which interface should respond to outside requests and which interface/gateway should be used to respond to those outside queries. Looking back at my initial email, I see that the way I worded my email makes it look like I was testing OUT from both interfaces,
but in fact, I was testing both out and inward with ICMP, and in both cases, that second interface failed to be able to talk to the outside world anywhere beyond it's gateway.
I've included the Interface configuration info below, to clarify your question of whether or not the two interfaces are in the
same subnet (they're not). I also included more in-depth listing of the policy routing information that was created (showing the two source routing table entries). But remember, before we even get to the question of why the source routing for the second interface doesn't seem to be working, there is the question of why
the two "mod_interface_route" commands ignored the IPv6 gateways altogether. Thank you for your help. Casey Russell Network Engineer Kansas Research and Education Network 2029 Becker Drive, Suite
282 Lawrence, KS 66047 On Mon, Jan 26, 2015 at 3:55 AM, Garnizov, Ivan (RRZE) <> wrote: Hi Casey, The “magic” of /opt/perfsonar_ps/toolkit/scripts/mod_interface_route is not purposed to fix routing.
The idea is to make server replies exit through the interface that has received inquiries. Meaning that your tests should not be performed on the host itself, but from a remote host. In order to make sure that the source routing works, you should make dump
on specific interface on the host you are configuring. The “problem” you see there is probably because you have both interfaces in the same subnet, which
means the routing is unable to determine, which is your desired path. It only sees that the second gateway is in the subnet of primary interface, therefore tries it through the primary interface, when initiating icmp tests. Please share IP settings’ details in order to clear off this possibility. Best regards, Ivan From:
[mailto:]
On Behalf Of Casey Russell Group, I'm setting up a new dual-interface system (the first for us), and am using the instructions provided at
http://docs.perfsonar.net/manage_dual_xface.html For clarity, the host will have 2 different physical interfaces in two separate subnets. Each interface will have IPv4 and IPv4 connectivity. The Basic IP and IPv6 setup goes simply
enough, I get the device base connectivity through the primary interface. And as you'd expect, the secondary interface can ping it's gateway, but can't talk to the outside world, because no policy (source routing) exists yet to give it a proper gateway. Then I ran the /opt/perfsonar_ps/toolkit/scripts/mod_interface_route script as suggested in the installation instructions. I ran it once for each interface (modifying the examples
to reflect the proper gateway info for my interfaces). Each command ran cleanly (at least appeared to). However, it appears to me that the script broke partway through executing each command and didn't properly build the routes for either interface.
After the commands are run. Here is the output of "ip rule list" That looks good. Looks like it installed the IPv4 gateway. However, I still can only ping the gateway for that network from the second Linux interface (em1). I have verified that
other IPs in that subnet can talk to the world (to rule out routing issues or an ACL at the router). Also, if you look at the IPv6 table after the mod_interface_route script has been run for either interface, you see that it didn't appear to modify the IPv6 tables at all. This
is why I suspect it quietly broke mid-operation. Does anyone know if there is a debug or "verbose" switch for that mod_interface_route script? (I tried the obvious -v --v) Thank you, Casey Russell Network Engineer Kansas Research and Education Network 2029 Becker Drive, Suite
282 Lawrence, KS 66047 |
- RE: [perfsonar-user] Dual Interface system, Garnizov, Ivan (RRZE), 04/14/2015
- Re: [perfsonar-user] Dual Interface system, Casey Russell, 04/14/2015
- RE: [perfsonar-user] Dual Interface system, Garnizov, Ivan (RRZE), 04/14/2015
- Re: [perfsonar-user] Dual Interface system, Casey Russell, 04/14/2015
- RE: [perfsonar-user] Dual Interface system, Garnizov, Ivan (RRZE), 04/14/2015
- Re: [perfsonar-user] Dual Interface system, Casey Russell, 04/14/2015
Archive powered by MHonArc 2.6.16.