Barcamp London 4 - Saturday afternoon

So I made it to the afternoon sessions, the first session that I attended was about MERB, which is an alternative Ruby based framework, lighter and faster than the Rails stack, which is claimed to achieve much higher performance than rails applications. Given that deployment and performance  are Rail's weakest points, the addition of MERB to the toolchest may stimulate development in this area. 

I caught up with some friends from Yahoo, and GCAP which is rapidly acquiring many of the best folks from the old Yahoo Business. Christian, Murray, and Mike shown above, who are definatly showing signs of  wear :-) 

Finally Simon Willison gave a fascinating talk about the Google APP Engine, which is a different take on the Elastic Computing Meme that is sweeping the net at the moment, providing some of the capabilities of the systems such as the Amazon EC2 system. 

Stay tuned for more ......

Barcamp London 4

Im at the first day of barcamplondon4, which is being held this week-end at GCAP's main offices in the west end of london. The "unconference" is running true to form, with delegates producing talks with wide ranging subject matter, such as "vitalising social networks", "scaling ruby on rails", "doing business in china" to "Australian wine". 

One of the rules of barcamp, is that anybody who attends must give a talk. GCAP is raising the stakes this year by recording all the presentation and making them available on its podcasting network, which is a first for barcamp. 

The other purpose of barcamp is the opportunity to network with people in the European web developer community, and barcamp attracts folks from all over Europe. This years event is particularly well attended by people from far shores, due to the concurrent running of "London Web Week" and events such as "@Media", making the week long trip to London an attractive proposition for those that want to cram in a whole years worth of developer conferences in one hit. 

I met a fascinating individual ( George Palmer) of idlasso.com who like myself is focused mainly on configuration and deployment issues, and we spent a enjoyable half hour exploring what the ultimate deployment architecture for small start-ups could look like, given the available and upcoming options. 

For myself, I am planning to do a talk on our own experiences of using some of these options to develop and deploy bejant.com, a nascent social network site, designed to introduce graduate students to potential employers. 

Anyway enough for now, I have to go and soak in more good material well i have access to such a concentration of braincells in one place. 

Freebsd on Amazon EC2?

I spotted a post on peat.org about an indicator that support for freebsd on amazon EC2 might be coming? 

This is apparently related to the new support for alternative OS's such as OpenSolaris , if anybody from amazon is listening then:

Please please please please please ........ support Freebsd on EC2, its stability and ease of use in server environments is second to none.  I would dump linux in a heartbeat if Freebsd support was available. 

Old Dog, New Tricks

Web Frameworks seem to be the fad of the month at the moment, I have looked at a number of them from the venerable Ruby On Rails, Cocoon, Symfony, CakePHP, Code Ignitor, Grovy and Grails. 

However I stumbled across a new one that really took me back this week. Cobol on Cogs  , yes that really is a web framework written in cobol.  All we need is Assembler on Acid and my week would be complete :-)

Doh........

Stupid Stupid Sun Linux VM installer %$&*%-up

Suns rpm Linux installer includes a number of unversioned resources for common installable packages.  So if you attempt to install something like "xml-commons-apis" with rpm or yum, the package manager tries to obsolete the unversioned package, and the only provider it can find is the jdk itself. 

So after installing this package, you suddenly discover your JDK has vaporised. 

So far there appears to be no known work around, other than to build a new sun jdk package from the src, which does not provide the unversioned packages. 

This fault has been around since feb 2007, and has not been fixed, I hit it when trying to install a red5 flash server, and could not get ant to run because it cant find the resource. 

Why is Java system configuration so damm hard? 

More details here:

Have the City of London Police been infiltrated by Scientology?

Yesterday the guardian ran a story about an anti-scientology demonstrator being prosecuted by the City of London Police (Note: Not the metropolitan police). The demonstrators "crime" was to use the word "cult" to describe Scientology on a placard during a demonstration. 

Aside from the obvious free speech issues, the article raises some interesting connections between the City of London Police force and the Church of Scientology, including senior officers accepting gifts from the church, and appearing in promotional videos. 

Scientology is a dangerous cult that exploits the more vulnerable members of society, whilst hiding behind laws designed to protect religious freedoms. Indeed in many countries, Uk and Germany notably, the state has stripped this cult of its religion status in order to prevent it exploiting the law, in the same way it attempts to exploit every other aspect of the society it disdains so much.  More European countries need to follow suit, to ensure that Scientology cannot establish the legal foothold it has in other regions. 

Serious questions need to be asked about the relationship between the City of London Police force and the Scientologists, and efforts should be made to determine just how far this insidious group has infiltrated an organisation that is supposed to serve in societies best interests. 

Amazon AWS - A practical experience - Part 1

Over the last few weeks I have been engaged in migrating a site I have been working on to the Amazon Web Services Environment. I have now got to the point where I feel I can start to write a series of posts about our experiences. This post is an introduction to this series.

Overview.

The site I have been working on migrating is Bejant.com, a LAMP based graduate employment social network that I have been working with for the last 3 months. The characteristics of Bejant are as follows

• PHP 5.2 based
• MySQL
• Apache 2.2
• Memcached
• Centos 5.0
  
• Swish-e Indexer (for search).
• Video Distribution and conversion. (ffmpeg).

The Amazon Web Services services used in this implementation are:

• Amazon EC2 - Elastic Computing Cloud
• Amazon S3 - Flexible Storage
• Amazon SQS - Message Queueing.

We are also evaluating the Amazon Simple DB as a means of persisting work storage between processes, but work in this area is at a very early phase.

The Runtime environment

Before we dive into the details of how we did this port, lets take a moment to list services that we are attempting to provide.

• 2 Front-end servers
• 2 Database servers
• 1 Test/QA server
• 1 Developer server
• 1 Video Processing Server
• 1 Utility server (ad server, mail-list manager, Feed processing pipeline).

We chose to use the RightScale management environment, which for a monthly fee provides Monitoring, Alerting and instance management and configuration.

I looked at a few other management tools such as Scalr and EC2PHP both of which did not provide enough capabilities to reasonably manage the cluster. It is indeed possible to roll your own, but we felt that RightScale gave us an edge and made creating this complex system setup easier and more maintainable.

RightScale provides the following:

• Replicated database solution
• Autoscaling
• Load balancing front ends
• Monitoring and alerting
• Multi-server clusters
• Log file consolidation
• Automated system administration
• Dynamic server configuration

Development Requirements

We decided that we wanted to create an environment that supports the full lifecycle of the Bejant.com development activity, which is predominantly SCRUMM based, to that end we wanted a production pipeline that moves releases from Development to Test to Live in an organised fashion. Bejant's sprint cycles operate on a approximately two week time line, during which a number of major and minor feature enhancements are introduced, alongside the usual maintenance and bugfixing activities that are normal for any development team. The Reason for the seperate Test enviroment is to isolate the QA folk from the day to day change that occures on a development system, and allow them to operate thier own database with known test accounts and data.

The challenge here is to make sure that the codebase and database schema are aligned on each stage of the pipeline, with a site such as Bejant that is undergoing rapid development, these elements are often quite different in each stage as new features are added and roled through to production.

To that end we decided that the system would effectively boot each stage from a subversion repository, which would hold branches that reflect the stages in the pipeline.

• The dev instances always boot from the trunk, and reflect the current state of the codebase.
  
• The Test instances boot from trunk, but are set to a particular revision that is deemed to be "in test", the test engineers can chose which revision to boot an instance from.
  
• The live system boots from a branch which represents a released product.
  

These "stages" are the same "enviroments" that are embodied in popular frameworks such as Ruby on Rails, Grails and Symfony.

In my next post we will look at some of the basics of AWS and the facilities it provides.