Java: Bad CPU id in executable

Here is a weird one:

Recently i have been doing a lot of java work, and many of the packages I use need to have Java_HOME set. So normally
i just add this line to my /etc/profile

JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home; export JAVA_HOME

Works just fine on my Macbook, and my recent 2009 Mac Mini. However i have an older 2006 Intel Mac Mini, the older Core Duo type, rather than the newer Core 2 Duo versions. Its been out of action for a few months due to a failed Hard Drive, but recently I revived it by buying a suitable hard drive from PCWorld and attacking the machine with a putty knife. (upgrading the ram whilst i was in there).

Everything seemed to be working fine, re-installed the OS, and I took the opportunity whilst i had a virgin system to document and automate the installation of my EC2 management scripts. Everything swam along fine until I actually tried to use the EC2 management scripts where upon I got the dreaded message "java: Bad CPU Id in executable" WTF....

Googling the error did not seem to help, so back to the drawing board, and take a microscope to the machine.

I checked the target directories, they are fine, in the "versions" directory under the java framework I can see 1.42, 1.50 and 1.60. so what is happening.

The first clue is that the java preferences applet does not list the 1.6.0 version as available.... Hmm why not????

The second clue is that switching to version 1.5.0 in the JAVA_HOME statement makes it all work.

The answer is that Apple never released a 32bit version of Java 1.6, but to rub salt into the wounds they load a perfectly non-functional 64bit version onto all 32bit only boxes. Ouch...

Come on Apple, there is no reason to limit java 1.6 to 64bit only, and even if there is, why on earth load a nonfunctional 64bit framework onto 32bit machines?

Shakes head in amazement.......

EBS, Flash drives for EC2 machines

Recently i had a bizarre situation where i had been running a load of stuff through Hadoop on one of our EC2 clusters, and the job had failed, but it failed in such a way that if i could save away the data, (and the logs for diagnostic purposes), and restart the cluster with some changed parameters i would be able to recover the data and carry on, having invested about $60 in machine time and hours of head scratching already on the failed run, I thought this was a good idea.

So no problem just use hadoop's distcp to move the data up to s3n://bucketname, hmmm that did not work out: OK, how about hadoop dfs -cp to a local directory of all the data and use s3cmd to move it to s3 storage, hmm that did not work out either, something odd is going on here.

Then it dawned on me that some of the files in the run where very large, as where the intermediate products, around 10G on average and there where lots of them. Now s3 has a limit of 2G for objects stored in its file system. I did not want to use the hadoop s3:// non native filesystem as that is difficult to verify if all the data had arrived safely, as nothing else would read it, so i had a brainwave, and successively mounted and unmounted an EBS volume onto each machine , saved away the relevant bits and pieces, and then moved onto the next box. All of our base EC2 images have xfs support wired into the system, as well as /mnt/ebs as a mountpoint for an attached ebs volume. So it was simply a case of using ElasticFox to attach the EBS volume to the instance and the issuing a "mount/mnt/ebs" and then "umount /mnt/ebs" once i was done.

So effectively using an EBS volume as a virtual flash-drive.

Fixing External monitors on the Aspire One and Fedora 10

I was recently forced to spend some time using my Aspire One as a primary development machine, having left my macbook PSU at work, and being too lazy to go all the way back in to retrieve it. 

So I thought, thats ok, I have this old monitor kicking around here, and a spare keyboard and mouse, just bang them into my Aspire One and off we go. But.......

Fedora 10 uses the new configless Xserver and the new Intel GEM enabled driver, a configless Xserver will happily run X without all the tedious fiddling with /etc/X11/xorg.conf. In fact if you take a peek in that directory you will see that there is no xorg.conf. 

So when I plugged my external monitor in, I was encouraged to see it display everything up to the login screen, logged in and was presented with a tiny 640x480 desktop on both monitors. 

Ok, no problem, pop up the Display properties dialog, set the sizes of the screens back to thier proper setup, 1024x600 for the builtin LCD, 1024x768 for the ext monitor. Press apply and ..... and .... and .... Nothing happened...  WHAT!!!!! 

OK it turns out that there is one vital piece of configuration that is missing in a configless Xserver boot, and thats the Virtual screen size.  You need a Virtual screen size that is bigger than the sum of both your monitor surfaces to run dual head (internal + external monitor). 

I tried everything, creating a minimal xorg.conf that just added that line, using sample xorg.confs for closely speced machines, nothing worked. And those that did, resulted in the disabling of DRI which produced syrupy screens on the external monitor. 

So the solution is to create an xorg.conf that is exactly what the autoconfig would produce, plus the extra line. 

So here are the steps

1. Reboot the Aspire and press RETURN when the initial bootloader screen comes up. 

2. press "a" and add a space then "3" to the end of the kernel boot line, press RETURN to boot the machine. 

3. The machine will boot to a command line instead of the usual graphical login screen. 

4. Login as root

5. Run the following

$cd /root
$Xorg -configure

The system will flash the screen and produce a file called xorg.conf.new in the current directory. This file contains all of the details that the configless startup would use.

$cp xorg.conf.new  /etc/X11/xorg.conf

Now locate the section at the end of that file that shows as below, and add the line indicated in

SubSection "Display"
   Viewport        0 0
   Depth          24
   Virtual        2048 2048           # Add this line in 
EndSubSection

6. Reboot the machine and you will now be able to reconfigure the monitors to support 1024x600 internal and 1024x768 external

Note: Because of limitations in the driver, if you use a value of more than 2048 in either dimension, it will disable DRI and the screen will become very slow again. This unfortunately limits the external display to 1024 width. If you want it to go wider then turn off the internal LCD panel in the Display Properties. This is being fixed in the next version of the driver, and this whole proceedure may become redundant by then as it will also support configless virtual spaces. 

Comparing Ubuntu 8.10 (Intrepid Ibix) and Fedora 10 on the Aspire One

I have been trying out several linux distributions on my Aspire One to find one that suites the machine best, I have tried everything, including a copy of PCBSD that was well "less than successful".

The two top runners are currently Fedora 10 (beta) and Ubuntu 8.10 (beta).

First a warning, Beta software is not for everybody, you can end up with a dead machine if you hit an issue with an update, and have to know how to recover your machine when that happens. You also have to have a good set of backups.

There is a superb tool called remastersys that creates boot able backups of not just your data, but your entire operating system, which I use in conjunction with an external FAT formatted USB drive.

One thing I have found is that trying to find a distro that supports the very recent hardware and chipsets found inside the Aspire One is hard, and I have narrowed my evaluations to those distros shipping the brand new 2.6.27 kernel or later, as that is the only one that seems to support the chipsets out of the box. I also need to have networkmanager 0.7, to support my Mobile USB broadband modem (a Huawei 169G).

I was using Ubuntu at first but needed to change to a RPM/YUM based system due to us using Centos 5 everywhere at work, and since I am building code that needs to be run and installed in that enviroment, the differences in the package managers was just too great for me to be comfortable with. (Yes I write software on my Aspire netbook, its quite capable of it, I was very surprised at how well Eclipse performs on this platform, almost as fast as my 2008 macbook).

Basics:

Installation of both distros was relatively easy, I used unetbootin to create a bootable 1G USB thumbdrive directly from the distributed ISO's and booted from that, I did not notice any issues with installation of either distro. Out of the box Fedora was slightly better in this respect, with a few caveats. Choose the gnome distros for each, as I have found that the newer KDE setups are somewhat less functional, in particular current KDE incarnations (4.1+) seem to have issues with saving settings.

Video:

Ubunto produced the more complete setup here, but only after I deleted the /etc/X11/xorg.conf file and allowed the new xorg 7.4 system to work its magic with configless boots.

glxgears turned in a performance of about 350 fps, which is fast enough to enable compiz desktop effects, however since that is just eye-candy its debatable whether it is worth enabling it on a device of this class. A minor irritation was that in order to support multiple screens, the screen resolution app has to create an xorg.conf with a virtual screen that encompasses both physical screens, and it initially gets it wrong which means that the external monitor resolution is low until you hand edit it to up the required size, I just doubled the dimensions in both directions and rebooted and was then able to select the 1280x1024 res I was looking for on my monitor.

Fedora was a mixed bag, it again needed the delete my xorg.conf trick, but try as I might I could not get it to properly support an external monitor at a reasonable resolution. However the performance of the driver in glxgears is significantly better, getting 550-600 fps. Im still trying to determine the reasons for this performance difference.

Wireless:

Due to the inclusion of networkmanager 0.7 in both distros, wireless was a doddle, both wifi and usb modem worked out of the box, however both distros suffered from the same issues re DHCP and wifi performance.

Every now and then they refused to acquire either a wired or wireless ip address due to dhcp timeouts, rebooting the machine seemed to clear the problem. The wifi is using the new ath5k driver for the Atheros chipset in both cases, and I have found that this driver seems to effect the sensitivity of of the wifi, with far lower signal strengths than under the older madwifi driver, and frequent dropouts and stalls.

Also under the madwifi driver there where a set of sysctls that would enable the wifi led which dont work on the ath5k driver, and I have not found any substitutes. The driver binds to the led_class module, and looking at the source has functions for enabling/disabling this mode, but I cant find any documentation on how to enable it.

laptop use:

Ubuntu has working suspend and resume, the sound sometimes does not restore properly coming out of sleep of hibernate but that is a minor annoyance, on Fedora both modes where a bust, resulting in a locked up machine requiring a hard reset.

Software:

I must admit my needs are probably different from average, I need to enable a full local LAMP stack and software development tools (yes it runs fine, and no the machine is not slow after doing so). The Aspire has dual 1.6Ghz cores, up to 1.5G of ram and a 120G HDD so its quite capable of handling this load. It should be noted that the spec of the machine is amost identical to the perfomance of an instance running on the Amazon EC2 cluster (1.6Ghz, 1.6G Ram, 160G HDD).

With Ubuntu setting up the stack was hard, on fedora this was a Breeze, Fedora even has a full Eclipse 3.4.x install in the repository, and has installable packages for eclipse PDT, subclipse and Xdebug. So my usual fight to get a working PHP dev enviroment working was eliminated.

I was able to setup up the entire machine for running our web app, including checking out the code from our subversion repository in under 30 mins, vs the 3-4 hour battle that I had with ubuntu. The Fedora packages even setup the correct SVN provider interfaces for subclipse which really impressed me.

I had some trouble setting up netbeans on Fedora, but mainly because it could not find the JRE directory, once that was sorted out, it installed fine. I tend to use Eclipse for PHP development, and Netbeans for C++ development as I have never really got on with the Eclipse CDT.

Overall:

Ubuntu is defiantly the more polished distro for general use, my specialised needs tend to lean me towards Fedora where im willing to put up with the shortcomings, I also like the faster more responsive feel to the fedora distro.

One final tip, if you are playing with beta software and hit issues, then engage with the community around the distro, they are normaly very responsive, and make sure that any problems you find are submitted as bug tickets, or they will never get fixed. Dont just sit back and wait for somebody else to report the problems...

Unbricking an aspire one

About 5 weeks ago i bought an Acer Aspire One, fantastic little machine, loaded it with Ubuntu (intrepid Ibix Alpha), 95% of the hardware works out of the box, even my 3UK 3G Modem, boy was I a happy camper.

I wanted the machine to run freemind so could take notes at FOWA which was last week.

Then suddenly disaster struck, I powered it up the week-end before the show only to be greeted with a blank screen and no activity at all. The machine was dead as a dodo.

So I packed it all back into the box and took it5 back to John Lewises in Reading where I had got it from originaly. And to give those guys their due, they where fantastic, they did not quibble, and swapped the machine out for a new one immediately without any hassle, at least one organisation knows about customer service (note they also include an extended 2 year warrentee for free with all items). I will be3 buying all my electronics from those guys in the future.

Anyway back to the story, I spent a frantic week-end reloading all my software and backups (yes I had them) onto the new machine, and headed off to the show. The machine performed fantastically, even managing to handle the wifi connections in the hall, where my colleagues EEE could not cut the mustard.

Then this evening on the train back home tonight, lightening struck twice. I powered down the AAO, realized I had not copied something I want off it onto my pen drive, and went to power it back up again, only to discover the machine had converted itself into a plastic brick again, totaly unresponsive to any prodding, engineer's taps or other incantations.

Dispondant at the thought of having to return it to JL with an explanation that "honest gov, it just broke again" . And negotiate the disdainful looks and insistence that I "must have done something to it", after all it is the second time... I resigned myself to having be without my AAO whilst JL investigated what abuse I had heaped on to the little beastie (again). All whilst feeling like a child molester, an abuser of young innocent netbooks.

However it turns out that this is a known problem, and the AAO even has a built in mechanism for fixing the problem, even if it is lying on its back with its metaphorical legs in the air. An off chance search of the net, looking for other lost souls with terminal aspire syndrome, hoping to find solace in the company of other unfortunates, and a chance to appeal my innocence to a more receptive group sharing this traumatic experience, turned up a post that offered a last chance hope of salvation.

Festooned with dire warnings about, following every step to the letter, and the dire consequences of not doing so, lay a page that made me once again aspire to get my aspire motoring again.

So The Aspire MAY drop its flashed bios occasionally, forcing it to emulate the common house brick, but it has a hardwired loader that will pull a copy of the bios off a usb pendrive and restore it to its former glory, even if the machine is exhibiting no other outward signs of life. The gory details can be found at at Macles Blog. Suffice to say i followed the recipe to the letter, waved the incantations in the air, mumbled the words of power, and breathed life back into my portable building material.

Waiting for the process to finish, and for the machine to restart has been the longest two minutes of my life, but to see the machine spring miraculous back to life, like Lazarus rising from the dead was a thrill worth raving about.

Phew......

Plaxo pulse, blog integration is a bit dubious

I just noticed that plaxo now aggregates content from your blogs and displays it inside the plaxo system. At fav.or.it we operate in this space so we have some experience in what is considered good etiquette here. 

1) We automatically scan the blog feeds for creative commons licensing information,  and if the license is missing or specifically denies commercial use we publish an extract of the article and a link to the original blog posting. Plaxo does NOT do this at all. 

2) We never show advertising against a blog post, the rights to commercial exploitation of content rests with the site that originates the content, Plaxo is taking liberties with peoples copyright by showing Google ads that earn them revenue against each blog post, many folks rely on professional revenue from pro-blogging to provide an income. 

3) Plaxo invites comments on each post which it shows against the listing, but unlike fav.or.it, they don't post the comments back to the originating blog, Plaxo is basically fuelling thier engagement on other peoples content, without contributing anything back to the originator. 

Poor show plaxo, you should sort this out!!!!. 

Hacking the Aspire One

I recent acquired an Acer Aspire One, this little device is fantastic, but is severely limited by the default software it is delivered with. After spending an evening opening it up and dismantling it, in order to stick in a spare 1G ram module that I had knocking around, to take it to 1.5G,  I then spent a further evening loading it with the latest release of Ubuntu, and wiping out the linspire linux it arrives with by default. 

Software updates include hacking the VodaPhone linux 3G card driver to work with my "3" 3G usb modem, spending hours tinkering with the madwifi wifi driver until it was able to connect to the network at home and at work. and playing with compiz and the Intel 945 video drivers until i had the 3D effects working at full speed.  

The Aspire One is an incredible machine, fast, small and compact with very very good performance. Small enough to slip into the poachers pocket in my barbour jacket. 

But the best hack I did to this machine, the one that transformed the device from an interesting toy into a usable portable machine, was the simplest, cheapest and fastest to implement. 

Sticking two rubber "feet" strips salvaged from the bottom of an old hard drive enclosure onto the two mouse buttons on the trackpad, has transformed the machine, I can now accuratly contol the mouse and click and double click, without having to feel around for the right spot to press down on. 

The Aspire One weights in at about £220, and delivers near desktop performance in a small portable package, with good 3+ hour battery life.