AMD64 / SATA RAID hangs during, after FC5 install

[dmarks]

I need some help getting FC5 working on an x64 machine with software RAID.

I recently invested in a shiny new dual-core Opteron server with the intent of using it as a database server. I experienced complete hangs when attempting to install off the FC5 DVD media (no mouse / capslock / etc). This happened at some point during the installation of packages to disk, often in the last 4 minutes of the install

I then attempted a text-mode install from the FC5 DVD, which was successful. However, the system is very unstable, hanging completely after a few minutes or hours of use. I'm about ready to try installing a different distribution of linux in the hopes that it will resolve whatever issues I have, but I thought I would check here first.

Questions:
- Has anyone successfully installed FC5 on this motherboard (H8DAR-T)?
- Any suggestions on alternate ways to configure software RAID with this hardware? I'm currently using the sata_mv driver and the on-board Marvell controller, however I'm concerned this might be the problem.

Hardware:
- 3 x Samsung 160GB SATA disks
- Supermicro H8DAR-T motherboard
- 1 x Opteron 265 CPU
- 2Gb RAM

Software:
- I'm trying to install the DVD FC5 x64 image

Dmesg output:
Bootdata ok (command line is ro root=/dev/md2 acpi=off)
Linux version 2.6.15-1.2054_FC5 (bhcompile@hs20-bc1-7.build.redhat.com) (gcc version 4.1.0 20060304 (Red Hat 4.1.0-3)) #1 SMP Tue Mar 14 15:48:20 EST 2006
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 000000007fff0000 (usable)
BIOS-e820: 000000007fff0000 - 000000007ffff000 (ACPI data)
BIOS-e820: 000000007ffff000 - 0000000080000000 (ACPI NVS)
BIOS-e820: 00000000ff780000 - 0000000100000000 (reserved)
Scanning NUMA topology in Northbridge 24
Number of nodes 1
Node 0 MemBase 0000000000000000 Limit 000000007fff0000
NUMA: Using 63 for the hash shift.
Using node hash shift of 63
Bootmem setup node 0 0000000000000000-000000007fff0000
On node 0 totalpages: 514834
DMA zone: 2785 pages, LIFO batch:0
DMA32 zone: 512049 pages, LIFO batch:31
Normal zone: 0 pages, LIFO batch:0
HighMem zone: 0 pages, LIFO batch:0
Intel MultiProcessor Specification v1.4
Virtual Wire compatibility mode.
OEM ID: MSI Product ID: RHAPSODY APIC at: 0xFEE00000
Processor #0 15:1 APIC version 16
Processor #1 15:1 APIC version 16
I/O APIC #2 Version 17 at 0xFEC00000.
I/O APIC #3 Version 17 at 0xFEBFE000.
I/O APIC #4 Version 17 at 0xFEBFF000.
Setting APIC routing to physical flat
Processors: 2
Allocating PCI resources starting at 88000000 (gap: 80000000:7f780000)
Checking aperture...
CPU 0: aperture @ 0 size 32 MB
No AGP bridge found
SMP: Allowing 2 CPUs, 0 hotplug CPUs
Built 1 zonelists
Kernel command line: ro root=/dev/md2 acpi=off
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 131072 bytes)
time.c: Using 1.193182 MHz WALL PIT GTOD PIT/TSC timer.
time.c: Detected 1804.118 MHz processor.
Console: colour VGA+ 80x25
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Memory: 2054868k/2097088k available (2307k kernel code, 41832k reserved, 1212k data, 196k init)
Calibrating delay using timer specific routine.. 3615.95 BogoMIPS (lpj=7231909)
Security Framework v1.0.0 initialized
SELinux: Initializing.
SELinux: Starting in permissive mode
selinux_register_security: Registering secondary module capability
Capability LSM initialized as secondary
Mount-cache hash table entries: 256
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU 0(2) -> Node 0 -> Core 0
Using IO-APIC 2
Using IO-APIC 3
Using IO-APIC 4
GSI 17 sharing vector 0x91 and IRQ 17
GSI 19 sharing vector 0x99 and IRQ 19
GSI 26 sharing vector 0xA1 and IRQ 26
GSI 33 sharing vector 0xA9 and IRQ 33
GSI 34 sharing vector 0xB1 and IRQ 34
Using local APIC timer interrupts.
result 12528605
Detected 12.528 MHz APIC timer.
Booting processor 1/2 APIC 0x1
Initializing CPU#1
Calibrating delay using timer specific routine.. 3608.47 BogoMIPS (lpj=7216940)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU 1(2) -> Node 0 -> Core 1
Dual Core AMD Opteron(tm) Processor 265 stepping 02
CPU 1: Syncing TSC to CPU 0.
CPU 1: synchronized TSC with CPU 0 (last diff 0 cycles, maxerr 487 cycles)
Brought up 2 CPUs

<snip due to posting length restrictions>

RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
AMD8111: IDE controller at PCI slot 0000:00:07.1
AMD8111: chipset revision 3
AMD8111: not 100% native mode: will probe irqs later
AMD8111: 0000:00:07.1 (rev 03) UDMA133 controller
ide0: BM-DMA at 0xffa0-0xffa7, BIOS settings: hdaio, hdbMA
ide1: BM-DMA at 0xffa8-0xffaf, BIOS settings: hdcio, hddio
Probing IDE interface ide0...
hdb: SONY DVD RW DW-Q58A, ATAPI CD/DVD-ROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
Probing IDE interface ide1...
Probing IDE interface ide1...
hdb: ATAPI 24X DVD-ROM DVD-R CD-R/RW drive, 2048kB Cache, UDMA(33)
Uniform CD-ROM driver Revision: 3.20
ide-floppy driver 0.99.newide
usbcore: registered new driver libusual
usbcore: registered new driver hiddev
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.6:USB HID core driver
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
NET: Registered protocol family 2
IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
TCP established hash table entries: 131072 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 9, 2097152 bytes)
TCP: Hash tables configured (established 131072 bind 65536)
TCP reno registered
TCP bic registered
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
powernow-k8: Found 2 AMD Athlon 64 / Opteron processors (version 1.60.0)
powernow-k8: MP systems not supported by PSB BIOS structure
powernow-k8: MP systems not supported by PSB BIOS structure
Freeing unused kernel memory: 196k freed
Write protecting the kernel read-only data: 432k
SCSI subsystem initialized
libata version 1.20 loaded.
sata_mv 0000:03:03.0: version 0.5
sata_mv 0000:03:03.0: Applying B2 workarounds to unknown rev
sata_mv 0000:03:03.0: 32 slots 4 ports unknown mode IRQ via INTx
ata1: SATA max UDMA/133 cmd 0x0 ctl 0xFFFFC200000A2120 bmdma 0x0 irq 26
ata2: SATA max UDMA/133 cmd 0x0 ctl 0xFFFFC200000A4120 bmdma 0x0 irq 26
ata3: SATA max UDMA/133 cmd 0x0 ctl 0xFFFFC200000A6120 bmdma 0x0 irq 26
ata4: SATA max UDMA/133 cmd 0x0 ctl 0xFFFFC200000A8120 bmdma 0x0 irq 26
Badness in __msleep at drivers/scsi/sata_mv.c:1730 (Not tainted)

Call Trace: <IRQ> <ffffffff8805757d>{:sata_mv:__mv_phy_reset+276}
<ffffffff8805727c>{:sata_mv:mv_channel_reset+137 } <ffffffff8805806e>{:sata_mv:mv_interrupt+607}
<ffffffff8015a52c>{handle_IRQ_event+41} <ffffffff8015a5f9>{__do_IRQ+156}
<ffffffff8010ce0d>{do_IRQ+59} <ffffffff8010ad72>{ret_from_intr+0} <EOI>
<ffffffff80339fce>{thread_return+0} <ffffffff8803e00f>{:libata:ata_check_status+15}
<ffffffff88057763>{:sata_mv:__mv_phy_reset+762} <ffffffff88057e0f>{:sata_mv:mv_interrupt+0}
<ffffffff8015aa09>{request_irq+139} <ffffffff88041b80>{:libata:ata_device_add+835}
<ffffffff802d1291>{pci_conf1_read+184} <ffffffff802d1291>{pci_conf1_read+184}
<ffffffff88058cea>{:sata_mv:mv_init_one+1531} <ffffffff8020cca8>{pci_device_probe+256}
<ffffffff8026d4a9>{driver_probe_device+82} <ffffffff8026d604>{__driver_attach+142}
<ffffffff8026d576>{__driver_attach+0} <ffffffff8026cea8>{bus_for_each_dev+67}
<ffffffff8026cb17>{bus_add_driver+118} <ffffffff8020ceee>{__pci_register_driver+142}
<ffffffff80157277>{stop_machine_run+58} <ffffffff8014d3b0>{sys_init_module+278}
<ffffffff8010a7aa>{system_call+126}
ata1: dev 0 cfg 49:2f00 82:346b 83:7fe9 84:4773 85:3469 86:3c01 87:4763 88:407f
ata1: dev 0 ATA-7, max UDMA/133, 321672960 sectors: LBA48
ata1: dev 0 configured for UDMA/133
scsi0 : sata_mv
ata2: dev 0 cfg 49:2f00 82:346b 83:7fe9 84:4773 85:3469 86:3c01 87:4763 88:407f
ata2: dev 0 ATA-7, max UDMA/133, 321672960 sectors: LBA48
ata2: dev 0 configured for UDMA/133
scsi1 : sata_mv
ata3: dev 0 cfg 49:2f00 82:346b 83:7fe9 84:4773 85:3469 86:3c01 87:4763 88:407f
ata3: dev 0 ATA-7, max UDMA/133, 321672960 sectors: LBA48
ata3: dev 0 configured for UDMA/133
scsi2 : sata_mv
ata4: no device found (phy stat 00000000)
scsi3 : sata_mv
Vendor: ATA Model: HDT722516DLA380 Rev: V43O
Type: Direct-Access ANSI SCSI revision: 05
SCSI device sda: 321672960 512-byte hdwr sectors (164697 MB)
sda: Write Protect is off
sda: Mode Sense: 00 3a 00 00
SCSI device sda: drive cache: write back
SCSI device sda: 321672960 512-byte hdwr sectors (164697 MB)
sda: Write Protect is off
sda: Mode Sense: 00 3a 00 00
SCSI device sda: drive cache: write back
sda: sda1 sda2 sda3
sd 0:0:0:0: Attached scsi disk sda

<snip due to posting length restrictions>

[dmarks]

Nobody replied, but I wanted to add some info in case others had the same issues. The issue appears to be the Adaptec HostRAID SATA controller. It isn't supported by sata_mv correctly, and there are no drivers released by adaptec for this controller on FC5. Additionally, the drivers for FC4 and RHEL appear to have bugs.

Avoid boards using the Adaptec HostRAID ar81xx controller -- they don't play nicely with Linux, and Adaptec won't release them as open source so others can fix the problem for them.

[msz]

No solution, just similar problem. Same hardware, FC4, installation goes fine (that's why my post did not go here) but then:

http://forums.fedoraforum.org/showthread.php?t=102306

[dmarks]

Thanks msz. I've installed CentOS 4.2 and it seems more stable than FC4. I just ran a series of IO benchmarks overnight, with success.

However, the strange Adaptec driver continues to cause loadavg=1.0 at idle.

One note: theoretically, the sata_mv driver should support the SATA controller without the Adaptec drivers. However, I experieced complete lockups when trying to use that driver on FC5.

At this point, I'm going to try getting a cheap PCI SATA board and connect the drives to that. If you haven't seen it, this page has loads of information about what is suppoted: http://linuxmafia.com/faq/Hardware/sata.html

[Maurizio]

I had a similar problem.
Try the option: linux nodmraid at boot time, that did it for me.

[Bluezman]

In reply of these postings: I have a similar problem
I want to install the 64 bit DVD version of Fedora 5.
Linux doesn't recognize the hard disk drive. The hard disk is an Hitachi HDT722516DLA380, an Hitachi Deskstar T7K250 (160 GB) (SATA).
Hoping for a solution other then replacing the hard disk I send this posting

[wdingus]

Most SuperMicro motherboards support Zero-Channel RAID with the addition of a RAID card, primarily made by Adaptec. Linux support for on-board SATA chipsets or Adaptec "HostRAID" has been very sketchy in my experience. I have a lot of SuperMicro boxes in production at customer sites with everything from RedHat 8 to Enterprise 4 and from FC3 to FC5 inhouse. All have ZCR cards, which additionally handle the RAID functions, offloading that from the primary CPU/OS.

http://www.supermicro.com/Aplus/moth...32/H8DAR-T.cfm
http://www.supermicro.com/products/a...AOC-LPZCR1.cfm

PS. Bluezman, it's probably not that Linux isn't recognizing the hard drive, rather the SATA chipset and/or controller. Here's a handy reference as to the status of support of various vendors SATA products with Linux: http://linux-ata.org/sata-status.html

[Bluezman]

The motherboard of the computer is a MSI RS482M4-ILD. The chipset is ATI® Radeon XPRESS 200 (RS482) and ATI® SB450 Chipset has also something to do with it or has to do with the SATA chipset or controller. Should I try to get linux support from ATI or MSI?

[msz]

Originally Posted by wdingus

Most SuperMicro motherboards support Zero-Channel RAID with the addition of a RAID card, primarily made by Adaptec. Linux support for on-board SATA chipsets or Adaptec "HostRAID" has been very sketchy in my experience. I have a lot of SuperMicro boxes in production at customer sites with everything from RedHat 8 to Enterprise 4 and from FC3 to FC5 inhouse. All have ZCR cards, which additionally handle the RAID functions, offloading that from the primary CPU/OS.

http://www.supermicro.com/Aplus/moth...32/H8DAR-T.cfm
http://www.supermicro.com/products/a...AOC-LPZCR1.cfm

PS. Bluezman, it's probably not that Linux isn't recognizing the hard drive, rather the SATA chipset and/or controller. Here's a handy reference as to the status of support of various vendors SATA products with Linux: http://linux-ata.org/sata-status.html

Wdingus, thanx for the hint. Does it mean that ZCR card is supported "natively" by Linux? Will it fit 1U enclosure?

[msz]

Originally Posted by dmarks

Thanks msz. I've installed CentOS 4.2 and it seems more stable than FC4. I just ran a series of IO benchmarks overnight, with success.

However, the strange Adaptec driver continues to cause loadavg=1.0 at idle.

One note: theoretically, the sata_mv driver should support the SATA controller without the Adaptec drivers. However, I experieced complete lockups when trying to use that driver on FC5.

At this point, I'm going to try getting a cheap PCI SATA board and connect the drives to that. If you haven't seen it, this page has loads of information about what is suppoted: http://linuxmafia.com/faq/Hardware/sata.html

Which driver for SATA? The same one or some different?

[wdingus]

The "native" aacraid driver works perfectly with this card... From the page at:

http://www.supermicro.com/Aplus/moth...32/H8DAR-T.cfm

is this:

Zero Channel RAID

* Support for Zero-Channel RAID with
* Low Profile, Supermicro All-In-One ZCR Card (AOC-LPZCR1)
* (Card not included)

And again, you can see a picture of the card to decide for yourself if it will fit in your enclosure here:

http://www.supermicro.com/products/a...AOC-LPZCR1.cfm

Actually I'm kind of surprised they built this board (which is obviously meant for 1U applications) to require a _regular_ card in a standard PCI slot. Most of the 1U boxes they do they have a special credit-card type slot for the ZCR, thereby not taking up the 1 or 2 available PCI slots. Like this one as an example:

http://www.supermicro.com/products/a...AOC-2025SA.cfm

From that same motherboard page, here's where the "Operating System Compatibility" link takes you:

http://www.supermicro.com/Aplus/supp...patibility.cfm

[Bluezman]

Thanks wdingus for your reply. In the meantime I did receive a DVD with Suse 10. I installed Suse linux without any problem. I don't know know why fedora linux doesn't work with the computer I tried it on.