Google just released AppEngine for 10,000 people.

A few Google engineers built a cool sample app called HuddleChat as a side project.

Jason Fried of 37signals fame says it's a direct rip off of Campfire.

Google stepped in and removed HuddleChat.

I'm a bit irritated by this. It's a bloody simple web based Chat application. If you think Campfire is the first to be implemented with the layout of having a chat screen on the left, online users and other gizmos on the right, then, uh, Jason, you live up to your arrogant reputation.

In the words of John Stossel: Give me a break!

Why is Google censoring? Why is Jason Fried being a twink? Why is anyone even listening to this clown? Maybe Jason should try getting real. Or maybe I'll have to reconsider what 37signals stands for. Sad.

I'm a pretty big Rails fan. Mostly because it's Ruby. More and more I'm starting to think that the Rails people are just too bloody cool for me and perhaps I should look at some of the new and up and coming Ruby frameworks like Merb (ok, not so new, but getting serious polish these days) or Wave. Who knows. I'm in a bad mood. App Engine is cool and these twinks at 37signals are just stealing publicity.

Today is 3/14

In two years we'll discover the fourth digit, but the fifth will baffle us for another five years after.

No it doesn't. But The Database Column would have you believe it.

A more fitting title to this may have been What Are These Guys Smoking?

Read the article here: MapReduce: A major step backwards.

Then take a deep breath and let it out the big "Huh?"

My first reaction to this is simple. Your typical database is based on perfectly structured data. Where the data doesn't fit neatly into the structure (schema) you transform it, often times using an ETL tool of sorts (thats Extract, Transform, Load).

Search data is anything but perfectly structured. Google indexes a whole lot of different document formats: HTML, PDF, word docs, excel files, and a whole lot more. This stuff doesn't exactly map to a neatly defined database schema does it?

MapReduce is a time tested proven and infinitely (well...) scalable method for building a resultset from a very large dataset. The neatest part perhaps is that the data that is mined can be in any number of different format stored in any media available. What does this translate into?

1. Cheap grunt servers
2. Cheap storage
3. Manageability
4. Scaling to the moon

Let's talk about that for a second. A low cost Linux server can be put into commission and last an enormous amount of time without every being upgraded. The data will be proliferated through many different systems providing an awesome set of redundant and low cost storage devices. Managing the servers is easy (in relative terms), seriously, they are grunts designed to do one task and do it very well. Upgrades should be minimal since they only have to change if the physical layout of the data changes, and it doesn't have to very often I bet. Scalability, no problem, plug in another thousand servers that are just begging for datasets to reduce and off they go.

Allow me to elaborate on the manageability front here. Say you have a linux server designed to dig through a crap-load of this data that you've been collecting through various means (spidering, general data loads, whatever). We're talking about a finely tuned $600 server (retail) that will kick ass at this job. More likely, we're talking about several thousand of these wicked machines. But, the only time you ever have to alter the innards of these beasts is if they fail or the shape of the data changes. With MapReduce you don't have to change the shape, just add a new generation of finely tuned $600 servers (retail) that that can see and process a new shape, and the new data. The old servers are just the aging rock stars, still jamming to the same tune, eventually they'll retire and the data will evolve. Ultimately the users question gets answered by everyone listening and reduced to the most relevant resultset.

Try that on any modern vendor provided database system and you'll probably find that it just can't be done that way. You have to have all of your data neatly ordered and ready to go. Show me a product that can evolve with your data, without having to migrate or transform that data's physical shape. I'll bet you can't. The reason you can't is because you have to funnel everything through their engine and it can't possibly be capable of knowing how all of the underlying data in a system like Google's. Google can add any data, in any shape it wants to their ginomrous clusters and the very nature of MapReduce screams "I don't care!" because the workers are doing the work.

So what were these guys who wrote this article writing about? Beats me. I think they were just trying to tell IT managers that Google is wrong and they know better. I think they are full of it if that's the case. Or perhaps they really just don't get it. I dunno.

Quite frankly I'm tired of hearing that we aren't in a recession, but we may be headed towards one. I don't buy the old definition of two consecutive quarters of declining GDP. That just doesn't sit well with me.

I want to know this instead. For every US dollar I spend, how much of that stays in the US economy?

I fill up the tank on my Saturn Aura, costs about $36 these days or more depending. How much of that $36 stays here?

When I buy three 2 liters of Diet Mt. Dew for $3.63, how much of that stays here?

Is there a measurement for this? I don't know. If there is, I'd like to see it. I'd like even more to hear about it in the news if that's possible.

Is there any wonder why young people are flocking to this guy? He had it right in '88 and he still has it right today.

Bottom line: Keep it simple you morons. Stop complicating our bloody lives.

Watch and see:

P.S. I just have nothing interesting to talk about in the realm of growing software. So enjoy!

Sometimes you are just in a really crappy mood.

And you just need something that makes it all make sense again:

It appears Douglas Adams was right on the money.

According to this article, the weight of life, the universe, and everything (well, almost) is 3x10^42. That's the number three with 42 zeros after it.

Rest in peace my friend.

Fortunately for us the Vogon destructor fleet must've fallen asleep on their way to obliterate the earth for the new hyperspace bypass. Whew.

And now for some Vogon Poetry...

Oh freddled gruntbuggly,
Thy micturations are to me
As plurdled gabbleblotchits
On a lurgid bee.
Groop, I implore thee, my foonting turlingdromes
And hooptiously drangle me
With crinkly bindlewurdles,
Or I will rend thee in the gobberwarts with my blurglecruncheon,
See if I don't!

I've done this at least a dozen times in VMware, OpenVZ, and Virtuozzo virtual servers. Every time I do it I start from scratch. It's silly. This time I'm writing down the steps so I don't forget.

Getting Started

Let's start with a Virtuozzo powered VPS running a clean installation of CentOS 4.4. We begin by removing a few things that we just don't need right now.

• Fetchmail
• SWSoft's Apache build
• SWSoft's Sendmail build
• The Pop3 server that we don't need

Remove what gets in the way

rpm -e fetchmail hspc-wwwroot
rpm -qa | grep httpd | xargs rpm -e
rpm -qa | grep sendmail | xargs rpm -e
rpm -qa|grep ^apr|xargs rpm -e
rpm -e popa3d
service xinetd restart

Install yum

In the virtual server I am on right now, yum is not installed. Odd. Let's install it:

rpm -i http://mirror.centos.org/centos/4.4/os/i386/CentOS/RPMS/yum-2.4.3-1.c4.noarch.rpm \
       http://mirror.centos.org/centos/4.4/os/i386/CentOS/RPMS/python-elementtree-1.2.6-4.2.1.i386.rpm \
       http://mirror.centos.org/centos/4.4/os/i386/CentOS/RPMS/python-sqlite-1.1.7-1.2.i386.rpm \
       http://mirror.centos.org/centos/4.4/os/i386/CentOS/RPMS/sqlite-3.3.3-1.2.i386.rpm \
       http://mirror.centos.org/centos/4.4/os/i386/CentOS/RPMS/sqlite-devel-3.3.3-1.2.i386.rpm \
       http://mirror.centos.org/centos/4.4/os/i386/CentOS/RPMS/python-urlgrabber-2.9.8-2.noarch.rpm

The CentOS-4 Testing repository

Now we want to add the c4-testing yum repository so we will install the latest Ruby rpm's. Or we can install from sources, but no point in doing that right now since c4-testing has Ruby 1.8.5.

cat << EOF > /etc/yum.repos.d/CentOS-Testing.repo
[c4-testing]
name=CentOS-4 Testing
baseurl=http://dev.centos.org/centos/\$releasever/testing/\$basearch/
enabled=1
gpgcheck=1
gpgkey=http://dev.centos.org/centos/RPM-GPG-KEY-CentOS-testing
EOF
yum -y update

Install Ruby, and Rails

As mentioned previously, you can install Ruby from sources or use the RPM provided in the CentOS repository. I use the CentOS built Ruby simply because it's already at the latest version (1.8.5 at this time) and it saves a little time.

Ruby

yum install -y ruby ruby-libs ruby-devel ruby-rdoc ruby-irb ruby-ri

Ruby Gems

wget http://rubyforge.org/frs/download.php/11289/rubygems-0.9.0.tgz
tar zxf rubygems-0.9.0.tgz
cd rubygems-0.9.0
ruby setup.rb

Rails, Mongrel, and Sqlite3-ruby

Rails, mongrel, and sqlite3-ruby:

gem install -y rails
gem install -y mongrel
gem install -y sqlite3-ruby

Apache and Subversion

And now we get to the tougher stuff. We need subversion if we plan to freeze_edge, which I always do during development. I also like to use Trac. We wont cover the installation of Trac today, but we'll make sure our environment is Trac friendly. More on that another time perhaps.

So let's get prepared for Subversion. We'll need the following items installed before we can build it the way we need it:

• Apache Portable Runtime (apr, apr-util
• The Apache Web Server (httpd)
• Neon
• SWIG

apr

Starting with apr and apr-util, we need to download, configure, build, and install from sources:

wget http://www.ip97.com/apache.org/apr/apr-1.2.8.tar.bz2
bzip2 -dc apr-1.2.8.tar.bz2 | tar xf -
cd apr-1.2.8
./configure && make && make install

apr-util

wget http://www.ip97.com/apache.org/apr/apr-util-1.2.8.tar.bz2
bzip2 -dc apr-util-1.2.8.tar.bz2 | tar xf -
cd apr-util-1.2.8
./configure --with-berkeley-db --with-apr=/usr/local/apr/bin/apr-1-config && make && make install

Apache Web Server

wget http://apache.downlod.in/httpd/httpd-2.2.3.tar.bz2
bzip2 -dc httpd-2.2.3.tar.bz2 | tar xf -
cd httpd-2.2.3
./configure -C \
            --prefix=/usr/local \
            --with-apr=/usr/local/apr/bin/apr-1-config \
            --with-apr-util=/usr/local/apr/bin/apu-1-config \
            --datadir=/var/www/home \
            --sysconfdir=/etc/httpd/conf \
            --enable-dav \
            --enable-suexec --with-suexec \
            --with-suexec-caller=apache \
            --with-suexec-docroot=/var/www \
            --with-suexec-logfile=/var/log/httpd/suexec.log \
            --with-suexec-bin=%{_sbindir}/suexec \
            --with-suexec-uidmin=500 --with-suexec-gidmin=500 \
            --with-devrandom \
            --enable-cache --enable-disk-cache --enable-mem-cache --enable-file-cache \
            --enable-ssl --with-ssl \
            --enable-deflate --enable-cgid \
            --enable-proxy --enable-proxy-connect \
            --enable-proxy-http --enable-proxy-ftp \
            --enable-proxy-balance && make && make install

Neon

wget http://www.webdav.org/neon/neon-0.25.5.tar.gz
tar zxf neon-0.25.5.tar.gz
cd neon-0.25.5
./configure --with-ssl=openssl && make && make install

SWIG

wget http://prdownloads.sourceforge.net/swig/swig-1.3.31.tar.gz
tar zxf swig-1.3.31.tar.gz
cd swig-1.3.31
./configure && make && make install

Subversion

wget http://subversion.tigris.org/downloads/subversion-1.4.2.tar.bz2
bzip2 -dc subversion-1.4.2.tar.bz2 | tar xf -
cd subversion-1.4.2
./configure --with-apxs=/usr/local/bin/apxs \
            --with-swig=/usr/local \
            --with-ssl \
            --with-neon=/usr/local \
            --with-apr=/usr/local/apr/bin \
            --with-apr-util=/usr/local/apr && make && make install

Good to go

At this point we're good to go. I usually install Vim7 right about now because I prefer to do all my coding in vi. CentOS 4 only has Vim6; Vim7 has newer rhtml and rb syntax files. Adding PostgreSQL or MySQL will complete your stack for a production environment. Sqlite3 is great for getting the ball rolling. Now it's time to be a Really Useful Engine and get working on your shiny new Rails app.

Good luck!

Think for a moment about the concept of the flamethrower. Okay? The flamethrower. Because we have them. Well, *we* don't have them, the army has them. That's right. We don't have any flamethrowers. I'd say we're fucked if we have to go up against the army, wouldn't you? But we have flamethrowers. And what this indicates to me, it means that at some point, some person said to himself, "Gee, I sure would like to set those people on fire over there. But I'm way to far away to get the job done. If only I had something that would throw flame on them." Well, it might have ended right there, but he mentioned it to his friend. His friend who was good with tools. And about a month later, he was back. "Hey, quite a concept!" WHHOOOOOOOOSSHHH! And of course the army heard about it, and they came around. "We'd like to buy about five hundred-thousand of them please. We have some people we'd like to throw flame on. Give us five hundred thousand and paint them dark brown. We don't want anyone to see them."

- George Carlin, 1984

s/people/code/g

So I'm stuck. I have to fix an application that was born broken. I mean the architecture, the design, the implementation, all of it. Broken. What does work works entirely by accident.

The application is a web application. Written for the ASP.NET platform with C# underpinnings. Not an entirely bad choice of platforms. Until you start looking under the hood. It's a procedural mess wrapped up in pseudo classes that act as nothing more than namespaces. Exceptions are silently absorbed and rarely managed. The so-called data access layer is nothing short of frightening.

The worst part? It's something that I helped build several years ago. I hold my head in shame. Kind of. In fact, several years ago I birthed a framework using ASP.NET and C# that I called "Epicenter." Epicenter was an ultra thin implementation of the Apache Cocoon framework that used XML quite liberally to define application flow and data access and used XSLT for rendering XHTML output. It was neat but it wasn't mature enough to be used in a production environment. Sso the choice was made to use the standard ASP.NET WebForms. I got frustrated and have long since lost the code for Epicenter. If this were on fark, I'd clearly deserve the Dumbass tag.

All I can say about this today is: Ooops, where's that bloody flamethrower?