After messing around with this some more, here's what I did to solve this:
I added biosdevname=0 to my kernel cmdline in grub. To do this I edited /etc/default/grub and added the parameter to GRUB_CMDLINE_LINUX. This simply makes the devices ethX instead of pXpY in Fedora 17. In the post I mentioned above was a reply that mentions simply removing the biosdevname package also makes it use ethX devices. I didn't try that solution but it's probably slightly more efficient! I'm fine with the ethX devices and I don't trust udev's renaming of them.
Next, I needed my e1000e built-in 4 x eth devices to be eth0,1,2,3 and my r8169 based add-on card to be eth4,5 to keep my sanity. Just letting the normal auto module load works but seemed like a race condition waiting to happen since the e1000e and 8169 modules are loading in parallel (evident from kernel dmesg showing printk's from both drivers interleaved when looking at the timestamps). I found that with systemd based stuff you can control the module load order for these by adding a file to /etc/modules-load.d with the desired order. I created /etc/modules-load.d/ethernet_drivers.conf and the content of it is:
Code:
# Load e1000e followed by r8169
e1000e
r8169
By looking at the handy kernel timestamps for e1000e and r8169 it's easy to see these are loaded serially instead of parallel -- and they are loaded earlier than the auto module loading since without the ethernet_drivers.conf file I would see timestamps from these drivers about 13 seconds in instead of 11.5 seconds through 12.249 like they are here:
Code:
[root@mmi-fw1 ~]# dmesg | grep e1000e
[ 11.512771] e1000e: Intel(R) PRO/1000 Network Driver - 1.9.5-k
[ 11.513273] e1000e: Copyright(c) 1999 - 2012 Intel Corporation.
[ 11.514419] e1000e 0000:05:00.0: Disabling ASPM L0s L1
[ 11.516273] e1000e 0000:05:00.0: PCI->APIC IRQ transform: INT C -> IRQ 19
[ 11.518163] e1000e 0000:05:00.0: (unregistered net_device): Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
[ 11.518515] e1000e 0000:05:00.0: irq 41 for MSI/MSI-X
[ 11.518574] e1000e 0000:05:00.0: irq 42 for MSI/MSI-X
[ 11.518629] e1000e 0000:05:00.0: irq 43 for MSI/MSI-X
[ 11.625944] e1000e 0000:05:00.0: eth0: (PCI Express:2.5GT/s:Width x1) 00:00:24:ce:e1:0c
[ 11.626393] e1000e 0000:05:00.0: eth0: Intel(R) PRO/1000 Network Connection
[ 11.628444] e1000e 0000:05:00.0: eth0: MAC: 3, PHY: 8, PBA No: FFFFFF-0FF
[ 11.629770] e1000e 0000:06:00.0: Disabling ASPM L0s L1
[ 11.631259] e1000e 0000:06:00.0: PCI->APIC IRQ transform: INT D -> IRQ 16
[ 11.632645] e1000e 0000:06:00.0: (unregistered net_device): Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
[ 11.633804] e1000e 0000:06:00.0: irq 44 for MSI/MSI-X
[ 11.633865] e1000e 0000:06:00.0: irq 45 for MSI/MSI-X
[ 11.634020] e1000e 0000:06:00.0: irq 46 for MSI/MSI-X
[ 11.745931] e1000e 0000:06:00.0: eth1: (PCI Express:2.5GT/s:Width x1) 00:00:24:ce:e1:0d
[ 11.753260] e1000e 0000:06:00.0: eth1: Intel(R) PRO/1000 Network Connection
[ 11.760343] e1000e 0000:06:00.0: eth1: MAC: 3, PHY: 8, PBA No: FFFFFF-0FF
[ 11.770372] e1000e 0000:0a:00.0: Disabling ASPM L0s L1
[ 11.782250] e1000e 0000:0a:00.0: PCI->APIC IRQ transform: INT C -> IRQ 16
[ 11.794547] e1000e 0000:0a:00.0: (unregistered net_device): Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
[ 11.806606] e1000e 0000:0a:00.0: irq 47 for MSI/MSI-X
[ 11.806691] e1000e 0000:0a:00.0: irq 48 for MSI/MSI-X
[ 11.806769] e1000e 0000:0a:00.0: irq 49 for MSI/MSI-X
[ 11.933857] e1000e 0000:0a:00.0: eth2: (PCI Express:2.5GT/s:Width x1) 00:00:24:ce:e1:0e
[ 11.934516] e1000e 0000:0a:00.0: eth2: Intel(R) PRO/1000 Network Connection
[ 11.935541] e1000e 0000:0a:00.0: eth2: MAC: 3, PHY: 8, PBA No: FFFFFF-0FF
[ 11.936311] e1000e 0000:0b:00.0: Disabling ASPM L0s L1
[ 11.939286] e1000e 0000:0b:00.0: PCI->APIC IRQ transform: INT D -> IRQ 17
[ 11.941921] e1000e 0000:0b:00.0: (unregistered net_device): Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
[ 11.943231] e1000e 0000:0b:00.0: irq 50 for MSI/MSI-X
[ 11.943308] e1000e 0000:0b:00.0: irq 51 for MSI/MSI-X
[ 11.943380] e1000e 0000:0b:00.0: irq 52 for MSI/MSI-X
[ 12.056010] e1000e 0000:0b:00.0: eth3: (PCI Express:2.5GT/s:Width x1) 00:00:24:ce:e1:0f
[ 12.063210] e1000e 0000:0b:00.0: eth3: Intel(R) PRO/1000 Network Connection
[ 12.071242] e1000e 0000:0b:00.0: eth3: MAC: 3, PHY: 8, PBA No: FFFFFF-0FF
[ 17.712476] e1000e: eth0 NIC Link is Up 100 Mbps Full Duplex, Flow Control: Rx/Tx
[ 17.713329] e1000e 0000:05:00.0: eth0: 10/100 speed: disabling TSO
[root@mmi-fw1 ~]# dmesg | grep r8169
[ 12.177190] r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
[ 12.190191] r8169 0000:0f:00.0: PCI->APIC IRQ transform: INT A -> IRQ 19
[ 12.197539] r8169 0000:0f:00.0: irq 53 for MSI/MSI-X
[ 12.199343] r8169 0000:0f:00.0: eth4: RTL8168c/8111c at 0xf7e36000, 00:e0:4c:68:d7:12, XID 1c4000c0 IRQ 53
[ 12.208214] r8169 0000:0f:00.0: eth4: jumbo features [frames: 6128 bytes, tx checksumming: ko]
[ 12.217300] r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
[ 12.231178] r8169 0000:10:00.0: PCI->APIC IRQ transform: INT B -> IRQ 16
[ 12.238335] r8169 0000:10:00.0: irq 54 for MSI/MSI-X
[ 12.239817] r8169 0000:10:00.0: eth5: RTL8168c/8111c at 0xf7e42000, 00:e0:4c:68:d6:e7, XID 1c4000c0 IRQ 54
[ 12.249186] r8169 0000:10:00.0: eth5: jumbo features [frames: 6128 bytes, tx checksumming: ko]
[ 16.612836] r8169 0000:0f:00.0: eth4: link down
[ 16.657749] r8169 0000:10:00.0: eth5: link down
The resulting network interfaces are exactly as I want them every time the box boots:
Code:
[root@mmi-fw1 ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.1.1.197 netmask 255.255.255.0 broadcast 10.1.1.255
inet6 fe80::200:24ff:fece:e10c prefixlen 64 scopeid 0x20<link>
ether 00:00:24:ce:e1:0c txqueuelen 1000 (Ethernet)
RX packets 2238 bytes 250972 (245.0 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 320 bytes 45544 (44.4 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 19 memory 0xa1000000-a1020000
eth1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 00:00:24:ce:e1:0d txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 16 memory 0xa2000000-a2020000
eth2: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 00:00:24:ce:e1:0e txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 16 memory 0xa3000000-a3020000
eth3: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 00:00:24:ce:e1:0f txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 17 memory 0xa4000000-a4020000
eth4: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 00:e0:4c:68:d7:12 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth5: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 00:e0:4c:68:d6:e7 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 16436
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 0 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
To finish this off I'll be assigning statics to each interface by using /etc/syconfig/network-scripts/ifcfg-ethX which I'll use the previous /etc/sysconfig/network-scripts/ifcfg-p1p1 as templates.