I have a Dell R710 running Debian Linux 6.0 (Squeeze). I installed a QLogic QLE8152 CNA in the PCI-E riser today and I have been trying to get the CNA's ethernet layer to come up after I put a fiber loop on it; I also scoped it with a light meter, and the SFP+ transceiver is getting the correct amount of light. The whole time I have been working on this problem, the lights on the CNA are blinking green at a rate of about 1 flash every 3 seconds.
When I plug the fiber into a Juniper EX4500 10GE in a rack next to it, the Juniper's link stays down as well.
I have to confess that this is my first wrangling with a CNA, so perhaps I'm doing something fundamentally wrong, but here is what I have found so far...
First the basics...
[mpenning@Finger ~]$ uname -a
Linux Finger 2.6.32-5-amd64 #1 SMP Mon Oct 3 03:59:20 UTC 2011 x86_64 GNU/Linux
[mpenning@Finger ~]$ cat /etc/issue
Debian GNU/Linux 6.0 \n \l
[mpenning@Finger ~]$
lspci -v shows that the card is properly installed (physically)
04:00.0 Ethernet controller: QLogic Corp. 10GbE Converged Network Adapter (TCP/IP Networking) (rev 02)
Subsystem: QLogic Corp. Device 017e
Flags: bus master, fast devsel, latency 0, IRQ 34
I/O ports at e000 [size=256]
Memory at df8f0000 (64-bit, non-prefetchable) [size=16K]
Memory at df900000 (64-bit, non-prefetchable) [size=1M]
Expansion ROM at df800000 [disabled] [size=256K]
Capabilities: [44] Power Management version 3
Capabilities: [4c] Express Endpoint, MSI 00
Capabilities: [88] MSI: Enable- Count=1/32 Maskable- 64bit+
Capabilities: [98] Vital Product Data
Capabilities: [a0] MSI-X: Enable+ Count=8 Masked-
Capabilities: [100] Advanced Error Reporting
Capabilities: [138] Device Serial Number 00-c0-dd-ff-fe-1c-53-b4
Capabilities: [144] Power Budgeting <?>
Kernel driver in use: qlge
The output from ethtool shows that eth4 is the QLE8152...
[mpenning@Finger ~]$ sudo ethtool eth4
Settings for eth4:
Supported ports: [ FIBRE ]
Supported link modes: 10000baseT/Full
Supports auto-negotiation: No
Advertised link modes: 10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: No
Speed: 10000Mb/s
Duplex: Full
Port: FIBRE
PHYAD: 0
Transceiver: external
Auto-negotiation: on
Current message level: 0x000060f7 (24823)
Link detected: no
[mpenning@Finger ~]$ sudo ethtool -i eth4
driver: qlge
version: v1.00.00-b3
firmware-version: v1.35.11
bus-info: 0000:04:00.0
[mpenning@Finger ~]$
Finally, I tried modprobe -r qlge; modprobe -r qla2xxx and then modprobe qla2xxx; modprobe qlge to kick the system again... I don't see a smoking gun in /var/log/messages...
Nov 17 19:46:21 finger kernel: [ 6212.298275] qlge 0000:04:00.1: PCI INT B disabled
Nov 17 19:46:22 finger kernel: [ 6213.779974] qlge 0000:04:00.0: PCI INT A disabled
Nov 17 19:46:33 finger kernel: [ 6224.554074] qla2xxx 0000:04:00.3: PCI INT D disabled
Nov 17 19:46:33 finger kernel: [ 6224.555322] qla2xxx 0000:04:00.2: PCI INT C disabled
Nov 17 19:46:54 finger kernel: [ 6245.625854] QLogic Fibre Channel HBA Driver: 8.03.01-k6
Nov 17 19:46:54 finger kernel: [ 6245.625888] qla2xxx 0000:04:00.2: PCI INT C -> GSI 35 (level, low) -> IRQ 35
Nov 17 19:46:54 finger kernel: [ 6245.626837] qla2xxx 0000:04:00.2: MSI-X vector count: 7
Nov 17 19:46:54 finger kernel: [ 6245.626841] qla2xxx 0000:04:00.2: Found an ISP8001, irq 35, iobase 0xffffc90012664000
Nov 17 19:46:54 finger kernel: [ 6245.627113] qla2xxx 0000:04:00.2: Configuring PCI space...
Nov 17 19:46:54 finger kernel: [ 6245.639429] qla2xxx 0000:04:00.2: Configure NVRAM parameters...
Nov 17 19:46:54 finger kernel: [ 6245.642597] qla2xxx 0000:04:00.2: Verifying loaded RISC code...
Nov 17 19:46:54 finger kernel: [ 6245.642708] qla2xxx 0000:04:00.2: FW: Loading from flash (a0000)...
Nov 17 19:46:55 finger kernel: [ 6246.273340] qla2xxx 0000:04:00.2: Allocated (64 KB) for FCE...
Nov 17 19:46:55 finger kernel: [ 6246.273401] qla2xxx 0000:04:00.2: Allocated (64 KB) for EFT...
Nov 17 19:46:55 finger kernel: [ 6246.273486] qla2xxx 0000:04:00.2: Allocated (1350 KB) for firmware dump...
Nov 17 19:46:55 finger kernel: [ 6246.273856] scsi9 : qla2xxx
Nov 17 19:46:55 finger kernel: [ 6246.274631] qla2xxx 0000:04:00.2:
Nov 17 19:46:55 finger kernel: [ 6246.274633] QLogic Fibre Channel HBA Driver: 8.03.01-k6
Nov 17 19:46:55 finger kernel: [ 6246.274634] QLogic QLE8152 - QLogic PCI-Express Dual Channel 10GbE CNA
Nov 17 19:46:55 finger kernel: [ 6246.274636] ISP8001: PCIe (5.0GT/s x4) @ 0000:04:00.2 hdma+, host#=9, fw=5.01.04 (8d4)
Nov 17 19:46:55 finger kernel: [ 6246.274666] qla2xxx 0000:04:00.3: PCI INT D -> GSI 37 (level, low) -> IRQ 37
Nov 17 19:46:55 finger kernel: [ 6246.274748] qla2xxx 0000:04:00.3: MSI-X vector count: 7
Nov 17 19:46:55 finger kernel: [ 6246.274751] qla2xxx 0000:04:00.3: Found an ISP8001, irq 37, iobase 0xffffc900125c4000
Nov 17 19:46:55 finger kernel: [ 6246.275098] qla2xxx 0000:04:00.3: Configuring PCI space...
Nov 17 19:46:55 finger kernel: [ 6246.287329] qla2xxx 0000:04:00.3: Configure NVRAM parameters...
Nov 17 19:46:55 finger kernel: [ 6246.290624] qla2xxx 0000:04:00.3: Verifying loaded RISC code...
Nov 17 19:46:55 finger kernel: [ 6246.290736] qla2xxx 0000:04:00.3: FW: Loading from flash (a0000)...
Nov 17 19:46:55 finger kernel: [ 6246.920204] qla2xxx 0000:04:00.3: Allocated (64 KB) for FCE...
Nov 17 19:46:55 finger kernel: [ 6246.920264] qla2xxx 0000:04:00.3: Allocated (64 KB) for EFT...
Nov 17 19:46:55 finger kernel: [ 6246.920345] qla2xxx 0000:04:00.3: Allocated (1350 KB) for firmware dump...
Nov 17 19:46:55 finger kernel: [ 6246.920749] scsi10 : qla2xxx
Nov 17 19:46:55 finger kernel: [ 6246.921715] qla2xxx 0000:04:00.3:
Nov 17 19:46:55 finger kernel: [ 6246.921716] QLogic Fibre Channel HBA Driver: 8.03.01-k6
Nov 17 19:46:55 finger kernel: [ 6246.921717] QLogic QLE8152 - QLogic PCI-Express Dual Channel 10GbE CNA
Nov 17 19:46:55 finger kernel: [ 6246.921719] ISP8001: PCIe (5.0GT/s x4) @ 0000:04:00.3 hdma+, host#=10, fw=5.01.04 (8d4)
Nov 17 19:46:58 finger kernel: [ 6249.519911] qlge 0000:04:00.0: PCI INT A -> GSI 34 (level, low) -> IRQ 34
Nov 17 19:46:58 finger kernel: [ 6249.533970] qlge 0000:04:00.0: QLogic 10 Gigabit PCI-E Ethernet Driver
Nov 17 19:46:58 finger kernel: [ 6249.533975] qlge 0000:04:00.0: Driver name: qlge, Version: v1.00.00-b3.
Nov 17 19:46:58 finger kernel: [ 6249.534856] qlge 0000:04:00.0: ql_display_dev_info: Function #0, Port 0, NIC Roll 0, NIC Rev = 1, XG Roll = 0, XG Rev = 1.
Nov 17 19:46:58 finger kernel: [ 6249.534860] qlge 0000:04:00.0: ql_display_dev_info: MAC address 00:c0:dd:1c:53:b4
Nov 17 19:46:58 finger kernel: [ 6249.534888] qlge 0000:04:00.1: PCI INT B -> GSI 44 (level, low) -> IRQ 44
Nov 17 19:46:58 finger kernel: [ 6249.549109] qlge 0000:04:00.1: ql_display_dev_info: Function #1, Port 1, NIC Roll 0, NIC Rev = 1, XG Roll = 0, XG Rev = 1.
Nov 17 19:46:58 finger kernel: [ 6249.549112] qlge 0000:04:00.1: ql_display_dev_info: MAC address 00:c0:dd:1c:53:b6
Any assistance getting a link up on this is greatly appreciated...