Out of the corner of my eye

Just writing a quick note to let anyone who reads this know about one of my colleagues blogs, Jen in Cambodia - she’s gone over there to work as a VSO recently and has already written some thought provoking pieces on the legacy of the Khmer Rouge and landmines. Please check it out.

visited 19 states (8.44%)

Courtesy of Douwe Osinga (go there to make your own map), via Matt Cutts.

Things I’ve learned about deploying a busy PHP application on Amazon EC2 WAMP/WIMP stacks
(the hard way)

In no particular order:

APC and __autoload

Rasmus said __autoload doesn’t play nice with APC.
This might have been true in 2006 when the blog post was written but it doesn’t seem so much following my tests (PHP 5.2.10 with APC 3.0.15). Our web application uses Ruby-esque MVC and URL dispatching with runtime context dependent class instantiation. There are around 28 controller and model classes. Including all of these every view incurred something like an additional 20-40% overhead over using __autoload. We’re using require_once for everything, too. APC saves you heaps of read operations - we went from something like 2k reads/second to a couple of hundred.

Memcache vs Memcached

Good luck using memcache’s binary protocol if you’re using Windows. Our (single so far) memcache box is instanced from Dustin’s memcached 1.4.0 on Ubuntu EC2 AMI. This runs like a dream. However, the current PHP memcache client is forked into memcache and memcached, and only memcache has a Windows build of the extension. Unfortunately only memcached lets you setOption OPT_BINARY_PROTOCOL. I gave myself a few hours on a Friday afternoon to try and build PHP and memcached in cygwin. Those few hours weren’t enough. It’s probably doable but it’s telling that nobody else seems to have done it yet.

Apache vs IIS

Under significant (but not really punishing) load, PHP on IIS gives “PHP has encountered an Access Violation at …” errors. Worker threads crash fairly frequently. We moved to Apache. Life is much simpler. Configuration of things like protected directories in the web root makes more sense - you move config from the server instance in IIS in that horrible pseudo-registry IIS uses to .htaccess files on your NFS web volume instead. CPU usage went down about 10-20% too…

EC2 Swap Confusion

EC2 can get horribly confused about its drives and where to put its swap file if you’re not careful. When making a new instance, follow these steps (if you’re using NFS EC2 volumes):
- Launch instance
- When instance is ready, reboot
- When instance has rebooted, attach volume
If you attach a volume too early, the instance might not find its own local d: drive and put its swap file on the tiny c: instead, resulting in 20 MB swap files. Not nice when you start getting low on physical RAM.

EC2 Monitoring (CloudWatch)

EC2 CloudWatch now has lovely Flash graphing and is very very nice. Use it.

XDebug, XProfile and CacheGrinding

XDebug with WinCacheGrind are very useful for profiling your PHP scripts and finding bottlenecks - however don’t use XDebug under heavy load or on a production box. Not because it kills your box (though it will slow it down some), but because it uses a ton of I/O writes to record its results which skews them towards implying everything is I/O bound. Try to look at percentages rather than absolute ms values to find bottlenecks / expensive loops etc.

Persistant MySQL connections

Seems to be a bit of FUD about these. They can help. We use them. As with anything like this be careful and read the documentation, and make sure you set sensible limits in php.ini and my.ini. Keep an eye on it hitting the ceiling too.

InnoDB vs MyISAM

InnoDB is now very mature and I don’t personally see any reason you’d use MyISAM anymore. Performance-wise you see  very little difference for many-read performance (MyISAM used to be faster if I remember) but get some very nice gains with tables that are frequent read/write because of row-level locking. You also get transactions and foreign key constraints (oooh, referential integrity!) should you need them (doing anything involving money or mission critical systems?). We recently moved from part MyISAM / part InnoDB (where our heavy write tables were InnoDB) to all InnoDB, meaning we could throw the majority of our RAM at Inno. Our db box has never been happier.

PHP sessions and uploads on EC2

Don’t put your sessions and temporary upload files in c:\windows\temp - put them in directories on your web root volume instead. It’s much faster.

Use local (private) IP’s for MySQL and memcache

Oops, initially we were using the public IP addresses of the db and memcache boxes to connect from the web servers. Silly me. Connecting to the 10.x.x.x IP means a few less hops, which means faster connections and less latency. Didn’t make a huge amount of difference for us, but is the sensible option…

Actually this is Ruby I guess, not even got as far as Rails! Main Rails page says “We recommend Ruby 1.8.7″ but there’s no installer for it. I think over the years I’ve had more installs of Rails fail than succeed. Back to PHP!

C:\ruby>gem install rails
Install required dependency actionpack? [Yn]
Install required dependency rack? [Yn]
Install required dependency fcgi? [Yn]
Building native extensions.  This could take a while…
ERROR:  While executing gem … (Gem::Installer::ExtensionBuildError)
ERROR: Failed to build gem native extension.

ruby extconf.rb install rails
checking for fcgiapp.h… no
checking for fastcgi/fcgiapp.h… no
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers.  Check the mkmf.log file for more
details.  You may need configuration options.

Provided configuration options:
–with-opt-dir
–without-opt-dir
–with-opt-include
–without-opt-include=${opt-dir}/include
–with-opt-lib
–without-opt-lib=${opt-dir}/lib
–with-make-prog
–srcdir=.
–curdir
–ruby=c:/ruby/bin/ruby
–with-fcgi-dir
–without-fcgi-dir
–with-fcgi-include
–without-fcgi-include=${fcgi-dir}/include
–with-fcgi-lib
–without-fcgi-lib=${fcgi-dir}/lib

Gem files will remain installed in c:/ruby/lib/ruby/gems/1.8/gems/fcgi-0.8.7 for
inspection.
Results logged to c:/ruby/lib/ruby/gems/1.8/gems/fcgi-0.8.7/ext/fcgi/gem_make.ou
t

So, I’ve taken the plunge and launched an internet radio station playing a selection of electronic dance music. Inspired by seeing my friends playing on SubbassFM, Afterdarkradio etc. and wanting to see our group of friends do something with our skills and determination to DJ, Liquifi Radio was born and launched on friday with a 5 hour show featuring dancefloor, liquid & tech dnb and house. Check us out and tune in sometime!

From the BBC:

He is expected to unveil soaring public borrowing and details of the worst recession since World War II ended.

Tax rises and spending cuts from 2011 are likely as Mr Darling sets out his plans to restore public finances.

Meanwhile, the International Monetary Fund (IMF) has admitted it made a mistake when it claimed the UK faced a £200bn bill for bailing out the banks.

A Treasury spokesman said the figure had been issued in the IMF’s Global Financial Stability Report in error.

Difficult times to try and balance your budget admittedly, but it has to be pointed out that Labour have been in power throughout the timeline of this boom and bust cycle. Headline of the Daily Mail (which I normally despise) yesterday was touting Labour’s failure of working class children in our education system, something that’s true and quite upsets me.

What are they going to do about this, other than throw more taxpayer money at the banks?

It’s that simple. Donate your TV Licence Fee to the Gaza DEC Appeal.

I have.

It’s time to drive the final nail into the coffin of laissez-faire capitalism by treating it like the discredited ideology it inarguably is. If not, the Dr. Frankensteins of the right will surely try to revive the monster and send it marauding through our economy once again.

…

We’ve got to do everything we can to make sure there will be no sequels to this political horror. The blame shifters cannot be allowed to make their case without the truth being pointed out at every turn. It’s time to relegate free market fundamentalists to the same standing as Marxist ideologues: intellectual curiosities occasionally trotted out as relics of a failed philosophy.

– Arianna Huffington, the Huffington Post

I could not agree more. We’ve been living under the illusion that our society is build upon regulated, centrist market policies, when in actual fact the banks and big corporations have robbed us blind, our politicians are complicit, and the whole damn ship is sinking. A return to economic conservatism that does not give Capital itself the reigns but instead works for the people. If a company isn’t contributing anything to society, but instead allows a tiny elite class to milk the world’s wealth before fleeing to their tax havens — then what good is it?

Do we defend globalisation, with its laissez-faire allowance of the exploitation of cheap mobile labour (see the construction workers shipped in with no rights and horrible pay to work on the building projects in Dubai for an extreme example), because of a liberal hang-wringing urge to shout about racism and xenophobism?

This is not nationalism, it is about the fair treatment of labour, whether the underpaid mobile labour from abroad or the passed over local workers at home. The only people winning in the stinking rat race of globalisation are the multi-nationals who get to treat people like slaves to make a cheap buck off their backs.

Welcome back to Victorian Britain, Mk II.

Internationalism versus Globalisation:

“While internationalism would celebrate the achievements, struggles and creativity of the poor, and the equal rights of all peoples of the world to develop in dignity, sufficiency and security, globalisation and global capitalism requires the humiliation of hundreds of millions of people and keeping them in constant insecurity, pitting them against one another in a competitive struggle for survival.

We are all globalisers now. The insistence on globalisation has eclipsed and usurped internationalism; indeed sometimes masquerades as if it were the same thing. It is time to rescue what internationalists have always worked for from the clutches of a rapaciously expansive and ultimately, colonising, globalisation.”

Update:

Just read this comment on the Guardian CIF:

Ex 3/ Burj Dubai built with imported “en masse” Indian and Filipino workers who live in same conditions, and have passports removed so they can’t go on strike and can be deported, and are typically not paid on time, while Dubai residents go skiing in their enclosed “Green” refrigirated ski run.

This is not “free movement of capital and labour”. This is the cynical tactic of “divide to rule” to break the rights of workers everywhere by shipping in foreign workers “en masse” and creating “closed systems”. This enables two birds to be killed with one stone: a) the imported workers are lost in a foreign environment, can’t communicate locally on the job market, and are frightened to assert their rights as human beings, and b) this enables workers protected in the UK to be conveniently circumvented.

 (Written by Klendathu)

I’ve just started reading Slaughterhouse-Five by Kurt Vonnegut, a kind of autobiography about his experience of the fire bombing of Dresden, “The Florence of the Elbe”, in the Second World War. Early on the book describes how he went to visit one of the men he served with, and finds his wife is angry with the idea of him writing the book.

Then she turned to me, let me see how angry she was, and that the anger was for me. She had been talking to herself, so what she said was a fragment of a much larger conversation. “You were just babies then!” she said.

“What?” I said.

“You were just babies in the war - like the ones upstairs!”

I nodded that this was true. We had been foolish virgins in the war, right at the end of childhood.

“But you’re not going to write it that way, are you.” This wasn’t a question. It was an accusation.

“I-I don’t know,” I said.

“Well, I know,” she said. “You’ll pretend you were men instead of babies, and you’ll be played in the movies by Frank Sinatra and John Wayne or some of those other glamorous, war-loving, dirty old men. And war will look just wonderful, so we’ll have a lot more of them. And they’ll be fought by babies like the babies upstairs.”

So then I understood. It was war that made her so angry. She didn’t want her babies or anybody else’s babies killed in wars. And she thought wars were partly encouraged by books and movies.

There have been 15 wars since 1945.

Dulce et decorum est pro patria mori, apparently.