I received a Lemote Fulong on August 21st, 2007. Here is everything that I have learned about it so far:
Opening the Box
Opening the box to the system, you will find that there is a small manual, a power cord with a Chinese plug on one end, a power inverter, and installation CD, and the Fulong unit itself. Fortunately the power inverter's AC plug is a standard C14 socket, so any cable that you may have that ends in a C13 plug will do. Voltage input on the inverter is a standard range of 100-240V and 50-60hz, with an input amperage of 1.7A. Output is 12VDC at 4.17A. Because of this it looks like it would be a very easy unit to replace if the power inverter goes bad.
Testing the machine with HPL requires a number of sub-programs to be installed.
The HPL.dat file I used is here: HPL.dat
A lot of help for getting this all together and working came from the following site:
Condensed HPL Outputs
As can be seen, the VSIPL libraries were very poor in HPL runs. Once the BLAS libraries (as tuned by ATLAS) were installed, the performance using the exact same HPL.dat file increased by 5.4 times! The next thing to do is to tune the other parameters of the dat file to see what we can get out of this machine.
As a contrast, running HPL in a virtual machine using VMWare on a quad core Intel Q6600 @ 2.4 Ghz, the VM only having 384mb of ram assigned to it, gave me a Gflop speed of 2.1 or so.
First on the list is mvapich2, which handles the MPI protocols needed in HPL.
Downloaded the latest and greatest version from http://mvapich.cse.ohio-state.edu/, mvapich2-1.0-beta.tar.gz.
Running Configure on this went well, so now I am going to time the compilation via make.
real 5m46.017s user 4m56.720s sys 0m40.552s
Also had to do configure with the following option:
Apparently threading is not a happy camper with this processor. Not that big a deal since in this case there is only one processor, but still....
Despite all of this, I was not able to get it to work, and since this is all really to get HPL working, I am going to fall back to MPICH2 instead.
Grabbed mpich2 from the following website: http://www-unix.mcs.anl.gov/mpi/mpich2/
Compiled with some amount of threading involved, though not the default:
Also found the major problem that was causing mvapich to break.... after editing /etc/hosts so that it should the local host as being the proper IP, things started to work properly.
My binary installation of it for debian installs is here:
Normally I would have used BLAS, but the BLAS website was down, so I am going to try using VSIPL instead. The libraries built fine and I installed them into /usr/local/lib.
It appears that the VSIPL library is not very efficient. At least, that is what I am hoping after the results I got using them. I have finally managed to get the BLAS libraries downloaded and am now compiling those. I expect that a speed test against them will be much better than the results I got with VSIPL.
BLAS and cBLAS were compiled, but they were not really helpful. Then I found ATLAS, which has all of that built in and does a ton of tuning as a part of the compile. This compiled cleanly, though it took a large number of hours to actually complete because of all of the testing that is performed.
The HPL makefile configuration that I used is Make.mipsel.
I recently changed the sources.list to point to the following:
deb http://mirrors.kernel.org/debian/ unstable main non-free contrib
I have now set off a apt-get update; apt-get upgrade, and will see what happens. This may break things, but in the meantime things appear to be alright.
This appears to have worked! Now I am going to do a dist-upgrade and cross my fingers!
The most recent versions of the X-server are not happy additions at this time. Don't try them. X-server has a tendency to crash. In order to recover from this I went back to the X-server that was originally installed by doing the following:
- Mount the install CD via NFS. I assume that the way to install from this disk is either via a USB cdrom drive or via some sort of tftp transfer.
- Once the disc is mounted, navigate to <mnt-point>/pool/main/x/xorg and use the following command:
dpkg --install * ../xserver-xorg-*/*
An odd thing about the g77 package is that it does not appear to have an actual executable called g77. There is one that is called g77-3.4, but there is no symbolic link to this for simple g77. I had to create one and throw it into /usr/local/bin.
Well, one of the dangers of doing something like this remotely is when the network manager gets reinstalled, and part of the re-installation is stopping that network manager:
Selecting previously deselected package network-manager. Preparing to replace network-manager 0.6.4-6 (using .../network-manager_0.6.4-8_mipsel.deb) ... Stopping network connection manager: NetworkManager
Poof! Away goes network, and thus my connection. So I am stuck until I get home and continue the install locally.
Lost a path to /usr/X11/bin and /usr/X11/sbin.
Lot of changes here, and I am not sure if this is a good thing. The old script was doing some things that the new script does not do at all, and I have to wonder if these are Fulong specific.
Out of the Box
By default out of the box, the Lemote Fulong comes installed with a modified version of debian. This is not really a bad thing, but it does have some odd things about it that are not part of a default install of debian (at least, not in my experience!)
- It is setup to boot straight into X. While this is not strange in and of itself, what is strange is that it boots straight into a user account with no password, using autologin via gdm.
- The root account is configured with a password. There is paperwork in the box with the machine detailing what the account passwords are and all, but if you are like me and cannot read Chinese, it might be a bit hard to decipher. As such, the password for the root account by default is "loongson".
- By default the network interface is configured for a static IP. I unfortunately deleted that portion before I got to writing this up, so I don't know what the exact IP is, but I do know it was one of the typical 192.168.0.x IP's.
- In other reports, I have seen mention that the fan on this machine was quite loud. In my experience, it is really not noticeable. Of course, the environment I have it in consists of a number of other machines that completely drown it out. Putting my ear up close to the machine, however, does elicit a slight hum, but it is nothing that would bother me. It would seem that Lemote has been listening to folks and has fixed this problem with the machine. Some of the links that support the loudness of the fan are listed here:
- In the PMOM, the boot device for a standard boot off of the harddrive is the following setting:
al = /dev/fs/ext2@wd0/boot/2.6.21
Other websites of interest
00:05.0 ISA bridge: VIA Technologies, Inc. VT82C686 [Apollo Super South] (rev 40) 00:05.1 IDE interface: VIA Technologies, Inc. VT82C586A/B/VT82C686/A/B/VT823x/A/C PIPC Bus Master IDE (rev 06) 00:05.2 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 1a) 00:05.3 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 1a) 00:05.4 Non-VGA unclassified device: VIA Technologies, Inc. VT82C686 [Apollo Super ACPI] (rev 40) 00:05.5 Multimedia audio controller: VIA Technologies, Inc. VT82C686 AC97 Audio Controller (rev 50) 00:05.6 Communication controller: VIA Technologies, Inc. AC'97 Modem Controller (rev 30) 00:06.0 VGA compatible controller: ATI Technologies Inc Radeon RV100 QZ [Radeon 7000/VE] 00:07.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) 00:08.0 USB Controller: NEC Corporation USB (rev 43) 00:08.1 USB Controller: NEC Corporation USB (rev 43) 00:08.2 USB Controller: NEC Corporation USB 2.0 (rev 04)
system type : lemote-fulong processor : 0 cpu model : ICT Loongson-2 V0.2 FPU V0.1 BogoMIPS : 444.41 wait instruction : no microsecond timers : yes tlb_entries : 64 extra interrupt vector : no hardware watchpoint : no ASEs implemented : VCED exceptions : not available VCEI exceptions : not available
Linux version 184.108.40.206lm32 (loongson@debian) (gcc version 4.1.2 20061028 (prerelease) (Debian 4.1.1-19)) #17 Thu Jun 14 14:18:13 CST 2007 busclock=66000000, cpuclock=668160000,memsize=256,highmemsize=768 CPU revision is: 00006302 FPU revision is: 00000501 Determined physical RAM map: memory: 0000000010000000 @ 0000000000000000 (usable) memory: 0000000010000000 @ 0000000010000000 (reserved) memory: 0000000030000000 @ 0000000020000000 (usable) On node 0 totalpages: 81920 DMA zone: 3 pages used for memmap DMA zone: 0 pages reserved DMA zone: 1021 pages, LIFO batch:0 Normal zone: 276 pages used for memmap Normal zone: 80620 pages, LIFO batch:7 Built 1 zonelists. Total pages: 81641 Kernel command line: console=tty root=/dev/hda1 Primary instruction cache 64kB, physically tagged, direct mapped, linesize 32 bytes. Primary data cache 64kB, 4-way, linesize 32 bytes. Unified secondary cache 512kB 4-way, linesize 32 bytes. Synthesized TLB refill handler (37 instructions). Synthesized TLB load handler fastpath (49 instructions). Synthesized TLB store handler fastpath (49 instructions). Synthesized TLB modify handler fastpath (48 instructions). arch init irq PID hash table entries: 4096 (order: 12, 32768 bytes) Using 334.080 MHz high precision timer. Console: colour dummy device 80x25 Dentry cache hash table entries: 262144 (order: 7, 2097152 bytes) Inode-cache hash table entries: 131072 (order: 6, 1048576 bytes) Memory: 1035040k/1048576k available (2933k kernel code, 12960k reserved, 967k data, 144k init, 0k highmem) Calibrating delay loop... 444.41 BogoMIPS (lpj=888832) Mount-cache hash table entries: 1024 Checking for the multiply/shift bug... no. Checking for the daddi bug... no. Checking for the daddiu bug... no. NET: Registered protocol family 16 registering PCI controller with io_map_base unset SCSI subsystem initialized usbcore: registered new interface driver usbfs usbcore: registered new interface driver hub usbcore: registered new device driver usb via686b fix: ISA bridge via686b fix: ISA bridge done via686b fix: IDE via686b fix: IDE done PCI quirk: region eee0-eeef claimed by vt82c686 SMB ac97 interrupt = 9 ac97 rev=80 Setting sub-vendor ID & device ID sub vendor-device id=47101005 NET: Registered protocol family 2 Time: MIPS clocksource has been installed. IP route cache hash table entries: 16384 (order: 3, 131072 bytes) TCP established hash table entries: 65536 (order: 6, 1048576 bytes) TCP bind hash table entries: 65536 (order: 5, 524288 bytes) TCP: Hash tables configured (established 65536 bind 65536) TCP reno registered fuse init (API version 7.8) io scheduler noop registered io scheduler anticipatory registered io scheduler deadline registered io scheduler cfq registered (default) radeonfb (0000:00:06.0): Invalid ROM signature ffff should be 0xaa55 xtal calculation failed: 26564 radeonfb: Used default PLL infos radeonfb: Reference=27.00 MHz (RefDiv=67) Memory=166.00 Mhz, System=166.00 MHz radeonfb: PLL min 12000 max 35000 radeonfb: Monitor 1 type CRT found radeonfb: Monitor 2 type no found Console: switching to colour frame buffer device 80x30 radeonfb (0000:00:06.0): ATI Radeon QZ rtc: SRM (post-2000) epoch (2000) detected Real Time Clock Driver v1.12ac Serial: 8250/16550 driver $Revision: 1.90 $ 2 ports, IRQ sharing disabled serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A loop: loaded (max 8 devices) 8139too Fast Ethernet driver 0.9.28 eth0: RealTek RTL8139 at 0x9000000016032000, 00:xx:xx:xx:xx:xx, IRQ 58 eth0: Identified 8139 chip type 'RTL-8100B/8139D' Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2 ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx VP_IDE: IDE controller at PCI slot 0000:00:05.1 VP_IDE: chipset revision 6 VP_IDE: not 100% native mode: will probe irqs later VP_IDE: VIA vt82c686b (rev 40) IDE UDMA100 controller on pci0000:00:05.1 ide0: BM-DMA at 0x5040-0x5047, BIOS settings: hda:pio, hdb:pio ide1: BM-DMA at 0x5048-0x504f, BIOS settings: hdc:pio, hdd:pio Probing IDE interface ide0... hda: HTS421240H9AT00, ATA DISK drive ide0 at 0x1f0-0x1f7,0x3f6 on irq 14 Probing IDE interface ide1... Probing IDE interface ide1... Probing IDE interface ide2... Probing IDE interface ide3... hda: max request size: 512KiB hda: 78140160 sectors (40007 MB) w/1570KiB Cache, CHS=16383/255/63, UDMA(33) hda: cache flushes supported hda: hda1 hda2 hda3 hda4 ehci_hcd 0000:00:08.2: EHCI Host Controller ehci_hcd 0000:00:08.2: new USB bus registered, assigned bus number 1 ehci_hcd 0000:00:08.2: irq 60, io mem 0x16032100 ehci_hcd 0000:00:08.2: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004 usb usb1: configuration #1 chosen from 1 choice hub 1-0:1.0: USB hub found hub 1-0:1.0: 4 ports detected ohci_hcd: 2006 August 04 USB 1.1 'Open' Host Controller (OHCI) Driver ohci_hcd 0000:00:08.0: OHCI Host Controller ohci_hcd 0000:00:08.0: new USB bus registered, assigned bus number 2 ohci_hcd 0000:00:08.0: irq 58, io mem 0x16030000 usb usb2: configuration #1 chosen from 1 choice hub 2-0:1.0: USB hub found hub 2-0:1.0: 2 ports detected ohci_hcd 0000:00:08.1: OHCI Host Controller ohci_hcd 0000:00:08.1: new USB bus registered, assigned bus number 3 ohci_hcd 0000:00:08.1: irq 59, io mem 0x16031000 usb 1-3: new high speed USB device using ehci_hcd and address 2 usb usb3: configuration #1 chosen from 1 choice hub 3-0:1.0: USB hub found hub 3-0:1.0: 2 ports detected usb 1-3: configuration #1 chosen from 1 choice usb 3-2: new full speed USB device using ohci_hcd and address 2 usb 3-2: configuration #1 chosen from 1 choice hub 3-2:1.0: USB hub found hub 3-2:1.0: 3 ports detected usb 3-2.1: new low speed USB device using ohci_hcd and address 3 usb 3-2.1: configuration #1 chosen from 1 choice usb 3-2.3: new full speed USB device using ohci_hcd and address 4 usb 3-2.3: configuration #1 chosen from 1 choice usbcore: registered new interface driver cdc_acm drivers/usb/class/cdc-acm.c: v0.25:USB Abstract Control Model driver for USB modems and ISDN adapters usbcore: registered new interface driver usblp drivers/usb/class/usblp.c: v0.13: USB Printer Device Class driver Initializing USB Mass Storage driver... usbcore: registered new interface driver usb-storage USB Mass Storage support registered. usbcore: registered new interface driver libusual serio: i8042 KBD port at 0x60,0x64 irq 1 serio: i8042 AUX port at 0x60,0x64 irq 12 mice: PS/2 mouse device common for all mice input: AT Translated Set 2 keyboard as /class/input/input0 atkbd.c: keyboard reset failed on isa0060/serio1 TCP cubic registered NET: Registered protocol family 1 NET: Registered protocol family 17 kjournald starting. Commit interval 5 seconds EXT3-fs: mounted filesystem with ordered data mode. VFS: Mounted root (ext3 filesystem) readonly. Freeing unused kernel memory: 144k freed USB Universal Host Controller Interface driver v3.0 uhci_hcd 0000:00:05.2: UHCI Host Controller uhci_hcd 0000:00:05.2: new USB bus registered, assigned bus number 4 uhci_hcd 0000:00:05.2: irq 10, io base 0x00005000 usb usb4: configuration #1 chosen from 1 choice hub 4-0:1.0: USB hub found hub 4-0:1.0: 2 ports detected usbcore: registered new interface driver hiddev uhci_hcd 0000:00:05.3: UHCI Host Controller uhci_hcd 0000:00:05.3: new USB bus registered, assigned bus number 5 uhci_hcd 0000:00:05.3: irq 11, io base 0x00005020 usb usb5: configuration #1 chosen from 1 choice hub 5-0:1.0: USB hub found hub 5-0:1.0: 2 ports detected input: Microsoft Microsoft 3-Button Mouse with IntelliEye(TM) as /class/input/input1 input: USB HID v1.10 Mouse [Microsoft Microsoft 3-Button Mouse with IntelliEye(TM)] on usb-0000:00:08.1-2.1 input: Mitsumi Electric Apple Extended USB Keyboard as /class/input/input2 input: USB HID v1.10 Keyboard [Mitsumi Electric Apple Extended USB Keyboard] on usb-0000:00:08.1-2.3 input: Mitsumi Electric Apple Extended USB Keyboard as /class/input/input3 input: USB HID v1.10 Device [Mitsumi Electric Apple Extended USB Keyboard] on usb-0000:00:08.1-2.3 usbcore: registered new interface driver usbhid drivers/usb/input/hid-core.c: v2.6:USB HID core driver PCI: Setting latency timer of device 0000:00:05.5 to 64 Adding 1023968k swap on /dev/hda3. Priority:-1 extents:1 across:1023968k EXT3 FS on hda1, internal journal kjournald starting. Commit interval 5 seconds EXT3 FS on hda2, internal journal EXT3-fs: mounted filesystem with ordered data mode. eth0: link up, 100Mbps, full-duplex, lpa 0x45E1