Lemote Fulong
I received a Lemote Fulong on August 21st, 2007. Here is everything that I have learned about it so far:
Opening the Box
Opening the box to the system, you will find that there is a small manual, a power cord with a Chinese plug on one end, a power inverter, and installation CD, and the Fulong unit itself. Fortunately the power inverter's AC plug is a standard C14 socket, so any cable that you may have that ends in a C13 plug will do. Voltage input on the inverter is a standard range of 100-240V and 50-60hz, with an input amperage of 1.7A. Output is 12VDC at 4.17A. Because of this it looks like it would be a very easy unit to replace if the power inverter goes bad.
HPL
Testing the machine with HPL requires a number of sub-programs to be installed.
The HPL.dat file I used is here: HPL.dat
Condensed HPL Outputs
| Library | NB | Gflops |
|---|---|---|
| VSIPL | 16 | .04225 |
| BLAS | 16 | .237 |
| BLAS | 32 | .3048 |
| BLAS | 40 |
As can be seen, the VSIPL libraries were very poor in HPL runs. Once the BLAS libraries (as tuned by ATLAS) were installed, the performance using the exact same HPL.dat file increased by 5.4 times! The next thing to do is to tune the other parameters of the dat file to see what we can get out of this machine.
mvapich
First on the list is mvapich2, which handles the MPI protocols needed in HPL.
Downloaded the latest and greatest version from http://mvapich.cse.ohio-state.edu/, mvapich2-1.0-beta.tar.gz.
Running Configure on this went well, so now I am going to time the compilation via make.
real 5m46.017s user 4m56.720s sys 0m40.552s
Also had to do configure with the following option:
./configure --enable-threads=single
Apparently threading is not a happy camper with this processor. Not that big a deal since in this case there is only one processor, but still....
Despite all of this, I was not able to get it to work, and since this is all really to get HPL working, I am going to fall back to MPICH2 instead.
mpich2
Grabbed mpich2 from the following website: http://www-unix.mcs.anl.gov/mpi/mpich2/
Compiled with some amount of threading involved, though not the default:
./configure --enable-threads=serialized
Also found the major problem that was causing mvapich to break.... after editing /etc/hosts so that it should the local host as being the proper IP, things started to work properly.
VSIPL
Normally I would have used BLAS, but the BLAS website was down, so I am going to try using VSIPL instead. The libraries built fine and I installed them into /usr/local/lib.
It appears that the VSIPL library is not very efficient. At least, that is what I am hoping after the results I got using them. I have finally managed to get the BLAS libraries downloaded and am now compiling those. I expect that a speed test against them will be much better than the results I got with VSIPL.
BLAS and cBLAS were compiled, but they were not really helpful. Then I found ATLAS, which has all of that built in and does a ton of tuning as a part of the compile. This compiled cleanly, though it took a large number of hours to actually complete because of all of the testing that is performed.
HPL Configuration
The HPL makefile configuration that I used is Make.mipsel.
sources.list
I recently changed the sources.list to point to the following:
deb http://mirrors.kernel.org/debian/ unstable main non-free contrib
I have now set off a apt-get update; apt-get upgrade, and will see what happens. This may break things, but in the meantime things appear to be alright.
This appears to have worked! Now I am going to do a dist-upgrade and cross my fingers!
Dangers
X-server
The most recent versions of the X-server are not happy additions at this time. Don't try them. X-server has a tendency to crash. In order to recover from this I went back to the X-server that was originally installed by doing the following:
- Mount the install CD via NFS. I assume that the way to install from this disk is either via a USB cdrom drive or via some sort of tftp transfer.
- Once the disc is mounted, navigate to <mnt-point>/pool/main/x/xorg and use the following command:
dpkg --install * ../xserver-xorg-*/*
gcc
An odd thing about the g77 package is that it does not appear to have an actual executable called g77. There is one that is called g77-3.4, but there is no symbolic link to this for simple g77. I had to create one and throw it into /usr/local/bin.
network-manager
Well, one of the dangers of doing something like this remotely is when the network manager gets reinstalled, and part of the re-installation is stopping that network manager:
Selecting previously deselected package network-manager. Preparing to replace network-manager 0.6.4-6 (using .../network-manager_0.6.4-8_mipsel.deb) ... Stopping network connection manager: NetworkManager
Poof! Away goes network, and thus my connection. So I am stuck until I get home and continue the install locally.
Changes
/etc/login.defs:
Lost a path to /usr/X11/bin and /usr/X11/sbin.
/etc/init.d/checkroot.sh:
Lot of changes here, and I am not sure if this is a good thing. The old script was doing some things that the new script does not do at all, and I have to wonder if these are Fulong specific.
Out of the Box
By default out of the box, the Lemote Fulong comes installed with a modified version of debian. This is not really a bad thing, but it does have some odd things about it that are not part of a default install of debian (at least, not in my experience!)
- It is setup to boot straight into X. While this is not strange in and of itself, what is strange is that it boots straight into a user account with no password, using autologin via gdm.
- The root account is configured with a password. There is paperwork in the box with the machine detailing what the account passwords are and all, but if you are like me and cannot read Chinese, it might be a bit hard to decipher. As such, the password for the root account by default is "loongson".
- By default the network interface is configured for a static IP. I unfortunately deleted that portion before I got to writing this up, so I don't know what the exact IP is, but I do know it was one of the typical 192.168.0.x IP's.
lspci output
00:05.0 ISA bridge: VIA Technologies, Inc. VT82C686 [Apollo Super South] (rev 40) 00:05.1 IDE interface: VIA Technologies, Inc. VT82C586A/B/VT82C686/A/B/VT823x/A/C PIPC Bus Master IDE (rev 06) 00:05.2 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 1a) 00:05.3 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 1a) 00:05.4 Non-VGA unclassified device: VIA Technologies, Inc. VT82C686 [Apollo Super ACPI] (rev 40) 00:05.5 Multimedia audio controller: VIA Technologies, Inc. VT82C686 AC97 Audio Controller (rev 50) 00:05.6 Communication controller: VIA Technologies, Inc. AC'97 Modem Controller (rev 30) 00:06.0 VGA compatible controller: ATI Technologies Inc Radeon RV100 QZ [Radeon 7000/VE] 00:07.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) 00:08.0 USB Controller: NEC Corporation USB (rev 43) 00:08.1 USB Controller: NEC Corporation USB (rev 43) 00:08.2 USB Controller: NEC Corporation USB 2.0 (rev 04)
/proc/cpuinfo output
system type : lemote-fulong processor : 0 cpu model : ICT Loongson-2 V0.2 FPU V0.1 BogoMIPS : 444.41 wait instruction : no microsecond timers : yes tlb_entries : 64 extra interrupt vector : no hardware watchpoint : no ASEs implemented : VCED exceptions : not available VCEI exceptions : not available
dmesg output
Linux version 2.6.21.3lm32 (loongson@debian) (gcc version 4.1.2 20061028 (prerelease) (Debian 4.1.1-19)) #17 Thu Jun 14 14:18:13 CST 2007
busclock=66000000, cpuclock=668160000,memsize=256,highmemsize=768
CPU revision is: 00006302
FPU revision is: 00000501
Determined physical RAM map:
memory: 0000000010000000 @ 0000000000000000 (usable)
memory: 0000000010000000 @ 0000000010000000 (reserved)
memory: 0000000030000000 @ 0000000020000000 (usable)
On node 0 totalpages: 81920
DMA zone: 3 pages used for memmap
DMA zone: 0 pages reserved
DMA zone: 1021 pages, LIFO batch:0
Normal zone: 276 pages used for memmap
Normal zone: 80620 pages, LIFO batch:7
Built 1 zonelists. Total pages: 81641
Kernel command line: console=tty root=/dev/hda1
Primary instruction cache 64kB, physically tagged, direct mapped, linesize 32 bytes.
Primary data cache 64kB, 4-way, linesize 32 bytes.
Unified secondary cache 512kB 4-way, linesize 32 bytes.
Synthesized TLB refill handler (37 instructions).
Synthesized TLB load handler fastpath (49 instructions).
Synthesized TLB store handler fastpath (49 instructions).
Synthesized TLB modify handler fastpath (48 instructions).
arch init irq
PID hash table entries: 4096 (order: 12, 32768 bytes)
Using 334.080 MHz high precision timer.
Console: colour dummy device 80x25
Dentry cache hash table entries: 262144 (order: 7, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 6, 1048576 bytes)
Memory: 1035040k/1048576k available (2933k kernel code, 12960k reserved, 967k data, 144k init, 0k highmem)
Calibrating delay loop... 444.41 BogoMIPS (lpj=888832)
Mount-cache hash table entries: 1024
Checking for the multiply/shift bug... no.
Checking for the daddi bug... no.
Checking for the daddiu bug... no.
NET: Registered protocol family 16
registering PCI controller with io_map_base unset
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
via686b fix: ISA bridge
via686b fix: ISA bridge done
via686b fix: IDE
via686b fix: IDE done
PCI quirk: region eee0-eeef claimed by vt82c686 SMB
ac97 interrupt = 9
ac97 rev=80
Setting sub-vendor ID & device ID
sub vendor-device id=47101005
NET: Registered protocol family 2
Time: MIPS clocksource has been installed.
IP route cache hash table entries: 16384 (order: 3, 131072 bytes)
TCP established hash table entries: 65536 (order: 6, 1048576 bytes)
TCP bind hash table entries: 65536 (order: 5, 524288 bytes)
TCP: Hash tables configured (established 65536 bind 65536)
TCP reno registered
fuse init (API version 7.8)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
radeonfb (0000:00:06.0): Invalid ROM signature ffff should be 0xaa55
xtal calculation failed: 26564
radeonfb: Used default PLL infos
radeonfb: Reference=27.00 MHz (RefDiv=67) Memory=166.00 Mhz, System=166.00 MHz
radeonfb: PLL min 12000 max 35000
radeonfb: Monitor 1 type CRT found
radeonfb: Monitor 2 type no found
Console: switching to colour frame buffer device 80x30
radeonfb (0000:00:06.0): ATI Radeon QZ
rtc: SRM (post-2000) epoch (2000) detected
Real Time Clock Driver v1.12ac
Serial: 8250/16550 driver $Revision: 1.90 $ 2 ports, IRQ sharing disabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
loop: loaded (max 8 devices)
8139too Fast Ethernet driver 0.9.28
eth0: RealTek RTL8139 at 0x9000000016032000, 00:xx:xx:xx:xx:xx, IRQ 58
eth0: Identified 8139 chip type 'RTL-8100B/8139D'
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
VP_IDE: IDE controller at PCI slot 0000:00:05.1
VP_IDE: chipset revision 6
VP_IDE: not 100% native mode: will probe irqs later
VP_IDE: VIA vt82c686b (rev 40) IDE UDMA100 controller on pci0000:00:05.1
ide0: BM-DMA at 0x5040-0x5047, BIOS settings: hda:pio, hdb:pio
ide1: BM-DMA at 0x5048-0x504f, BIOS settings: hdc:pio, hdd:pio
Probing IDE interface ide0...
hda: HTS421240H9AT00, ATA DISK drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
Probing IDE interface ide1...
Probing IDE interface ide1...
Probing IDE interface ide2...
Probing IDE interface ide3...
hda: max request size: 512KiB
hda: 78140160 sectors (40007 MB) w/1570KiB Cache, CHS=16383/255/63, UDMA(33)
hda: cache flushes supported
hda: hda1 hda2 hda3 hda4
ehci_hcd 0000:00:08.2: EHCI Host Controller
ehci_hcd 0000:00:08.2: new USB bus registered, assigned bus number 1
ehci_hcd 0000:00:08.2: irq 60, io mem 0x16032100
ehci_hcd 0000:00:08.2: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 4 ports detected
ohci_hcd: 2006 August 04 USB 1.1 'Open' Host Controller (OHCI) Driver
ohci_hcd 0000:00:08.0: OHCI Host Controller
ohci_hcd 0000:00:08.0: new USB bus registered, assigned bus number 2
ohci_hcd 0000:00:08.0: irq 58, io mem 0x16030000
usb usb2: configuration #1 chosen from 1 choice
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 2 ports detected
ohci_hcd 0000:00:08.1: OHCI Host Controller
ohci_hcd 0000:00:08.1: new USB bus registered, assigned bus number 3
ohci_hcd 0000:00:08.1: irq 59, io mem 0x16031000
usb 1-3: new high speed USB device using ehci_hcd and address 2
usb usb3: configuration #1 chosen from 1 choice
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 2 ports detected
usb 1-3: configuration #1 chosen from 1 choice
usb 3-2: new full speed USB device using ohci_hcd and address 2
usb 3-2: configuration #1 chosen from 1 choice
hub 3-2:1.0: USB hub found
hub 3-2:1.0: 3 ports detected
usb 3-2.1: new low speed USB device using ohci_hcd and address 3
usb 3-2.1: configuration #1 chosen from 1 choice
usb 3-2.3: new full speed USB device using ohci_hcd and address 4
usb 3-2.3: configuration #1 chosen from 1 choice
usbcore: registered new interface driver cdc_acm
drivers/usb/class/cdc-acm.c: v0.25:USB Abstract Control Model driver for USB modems and ISDN adapters
usbcore: registered new interface driver usblp
drivers/usb/class/usblp.c: v0.13: USB Printer Device Class driver
Initializing USB Mass Storage driver...
usbcore: registered new interface driver usb-storage
USB Mass Storage support registered.
usbcore: registered new interface driver libusual
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
input: AT Translated Set 2 keyboard as /class/input/input0
atkbd.c: keyboard reset failed on isa0060/serio1
TCP cubic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
kjournald starting. Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 144k freed
USB Universal Host Controller Interface driver v3.0
uhci_hcd 0000:00:05.2: UHCI Host Controller
uhci_hcd 0000:00:05.2: new USB bus registered, assigned bus number 4
uhci_hcd 0000:00:05.2: irq 10, io base 0x00005000
usb usb4: configuration #1 chosen from 1 choice
hub 4-0:1.0: USB hub found
hub 4-0:1.0: 2 ports detected
usbcore: registered new interface driver hiddev
uhci_hcd 0000:00:05.3: UHCI Host Controller
uhci_hcd 0000:00:05.3: new USB bus registered, assigned bus number 5
uhci_hcd 0000:00:05.3: irq 11, io base 0x00005020
usb usb5: configuration #1 chosen from 1 choice
hub 5-0:1.0: USB hub found
hub 5-0:1.0: 2 ports detected
input: Microsoft Microsoft 3-Button Mouse with IntelliEye(TM) as /class/input/input1
input: USB HID v1.10 Mouse [Microsoft Microsoft 3-Button Mouse with IntelliEye(TM)] on usb-0000:00:08.1-2.1
input: Mitsumi Electric Apple Extended USB Keyboard as /class/input/input2
input: USB HID v1.10 Keyboard [Mitsumi Electric Apple Extended USB Keyboard] on usb-0000:00:08.1-2.3
input: Mitsumi Electric Apple Extended USB Keyboard as /class/input/input3
input: USB HID v1.10 Device [Mitsumi Electric Apple Extended USB Keyboard] on usb-0000:00:08.1-2.3
usbcore: registered new interface driver usbhid
drivers/usb/input/hid-core.c: v2.6:USB HID core driver
PCI: Setting latency timer of device 0000:00:05.5 to 64
Adding 1023968k swap on /dev/hda3. Priority:-1 extents:1 across:1023968k
EXT3 FS on hda1, internal journal
kjournald starting. Commit interval 5 seconds
EXT3 FS on hda2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
