makefile.com - Home

Silly Twink

2008-04-09T19:56:00Z

Google just released AppEngine for 10,000 people.

A few Google engineers built a cool sample app called HuddleChat as a side project.

Jason Fried of 37signals fame says it's a direct rip off of Campfire.

Google stepped in and removed HuddleChat.

I'm a bit irritated by this. It's a bloody simple web based Chat application. If you think Campfire is the first to be implemented with the layout of having a chat screen on the left, online users and other gizmos on the right, then, uh, Jason, you live up to your arrogant reputation.

In the words of John Stossel: Give me a break!

Why is Google censoring? Why is Jason Fried being a twink? Why is anyone even listening to this clown? Maybe Jason should try getting real. Or maybe I'll have to reconsider what 37signals stands for. Sad.

I'm a pretty big Rails fan. Mostly because it's Ruby. More and more I'm starting to think that the Rails people are just too bloody cool for me and perhaps I should look at some of the new and up and coming Ruby frameworks like Merb (ok, not so new, but getting serious polish these days) or Wave. Who knows. I'm in a bad mood. App Engine is cool and these twinks at 37signals are just stealing publicity.

Happy PI day!

2008-03-14T04:00:00Z

Today is 3/14

In two years we'll discover the fourth digit, but the fifth will baffle us for another five years after.

Better C# - Anonymous Delegates

2008-01-24T22:41:00Z

C# has been kind of in and out of my life since its inception way back when. At one point I really liked it, then I started to despise it because of how garbage collection works and how easy it is to leak resources when you aren't proactively destroying object instances.

Luckily the IDispose interface is available to take care of such things, and you can often achieve the same effect through use of a try/catch/finally clause, ensuring resources are released during a visit to the finally block.

I generally use the try/catch/finally approach because I find it more obvious and I don't have to wrap up existing classes with an IDispose implementation.

Another one of my frequent gripes about C# is how verbose it is. The latest generation of C# will introduce a form a type inference that will improve on this in a big way. For now there are still some things we can do to make our active code smaller and more manageable.

Today I used anonymous delegates to clean up some code and remove a lot of repeated junk.

The pattern I had went something like this:

public static string create(string name)
{
  IDbConnection db = new System.Data.SqlClient.SqlConnection();
  try
  {
    db.ConnectionString = connect_string_for(null);
    db.Open();
    IDbCommand command = db.CreateCommand();
    command.CommandText = string.Format("CREATE DATABASE {0}", name);
    command.ExecuteNonQuery();
  }
  catch (System.Exception x)
  {
    return x.Message;
  }
  finally
  {
    if (db != null && Tools.included_in<ConnectionState>(db.State, Tools.db_open_states))
    {
      db.Close();
    }       
  }
}

Using anonymous delegates (also known as anonymous methods) I was able to extract the repetitive junk, namely the try, catch, and finally stuff plus the db connection piece into a reusable method that named with_db_for that took the name of the database and the delegate for using the database. Now that code looks like this:

public string create()
{
  string result = Tools.with_db_for(null, delegate(IDbConnection db)
  {
    IDbCommand cmd = db.CreateCommand();
    cmd.CommandText = string.Format("CREATE DATABASE {0}", dbname);
    cmd.ExecuteNonQuery();
    return "yes";
  });

  dropped = result == "yes";
  return result;
}

Ahhh, much better.

I'm going to have to keep exploring this. Anonymous delegates are lexical closures which is very satisfying, but they are also strongly typed which is weird for such constructs. I am very pleased with this.

The future looks even better. I am really interested in type inference and many of the other benefits of the next generation of C#, and perhaps my mind will change a little and I'll actually start enjoying it once more.

But never as much as I enjoy Ruby.

MapReduce Sucks!

2008-01-19T02:25:00Z

No it doesn't. But The Database Column would have you believe it.

A more fitting title to this may have been What Are These Guys Smoking?

Read the article here: MapReduce: A major step backwards.

Then take a deep breath and let it out the big "Huh?"

My first reaction to this is simple. Your typical database is based on perfectly structured data. Where the data doesn't fit neatly into the structure (schema) you transform it, often times using an ETL tool of sorts (thats Extract, Transform, Load).

Search data is anything but perfectly structured. Google indexes a whole lot of different document formats: HTML, PDF, word docs, excel files, and a whole lot more. This stuff doesn't exactly map to a neatly defined database schema does it?

MapReduce is a time tested proven and infinitely (well...) scalable method for building a resultset from a very large dataset. The neatest part perhaps is that the data that is mined can be in any number of different format stored in any media available. What does this translate into?

Cheap grunt servers
Cheap storage
Manageability
Scaling to the moon

Let's talk about that for a second. A low cost Linux server can be put into commission and last an enormous amount of time without every being upgraded. The data will be proliferated through many different systems providing an awesome set of redundant and low cost storage devices. Managing the servers is easy (in relative terms), seriously, they are grunts designed to do one task and do it very well. Upgrades should be minimal since they only have to change if the physical layout of the data changes, and it doesn't have to very often I bet. Scalability, no problem, plug in another thousand servers that are just begging for datasets to reduce and off they go.

Allow me to elaborate on the manageability front here. Say you have a linux server designed to dig through a crap-load of this data that you've been collecting through various means (spidering, general data loads, whatever). We're talking about a finely tuned $600 server (retail) that will kick ass at this job. More likely, we're talking about several thousand of these wicked machines. But, the only time you ever have to alter the innards of these beasts is if they fail or the shape of the data changes. With MapReduce you don't have to change the shape, just add a new generation of finely tuned $600 servers (retail) that that can see and process a new shape, and the new data. The old servers are just the aging rock stars, still jamming to the same tune, eventually they'll retire and the data will evolve. Ultimately the users question gets answered by everyone listening and reduced to the most relevant resultset.

Try that on any modern vendor provided database system and you'll probably find that it just can't be done that way. You have to have all of your data neatly ordered and ready to go. Show me a product that can evolve with your data, without having to migrate or transform that data's physical shape. I'll bet you can't. The reason you can't is because you have to funnel everything through their engine and it can't possibly be capable of knowing how all of the underlying data in a system like Google's. Google can add any data, in any shape it wants to their ginomrous clusters and the very nature of MapReduce screams "I don't care!" because the workers are doing the work.

So what were these guys who wrote this article writing about? Beats me. I think they were just trying to tell IT managers that Google is wrong and they know better. I think they are full of it if that's the case. Or perhaps they really just don't get it. I dunno.

US economy and a simple question

2008-01-18T05:03:00Z

Quite frankly I'm tired of hearing that we aren't in a recession, but we may be headed towards one. I don't buy the old definition of two consecutive quarters of declining GDP. That just doesn't sit well with me.

I want to know this instead. For every US dollar I spend, how much of that stays in the US economy?

I fill up the tank on my Saturn Aura, costs about $36 these days or more depending. How much of that $36 stays here?

When I buy three 2 liters of Diet Mt. Dew for $3.63, how much of that stays here?

Is there a measurement for this? I don't know. If there is, I'd like to see it. I'd like even more to hear about it in the news if that's possible.

Kozlov in Detroit

2008-01-16T04:19:00Z

It was cool to see Slava Kozlov back in Detroit finally. We've waited all season for this game.

My wife was pleased that both the Thrashers won (5-1 in an odd Wings loss) and that Kozzy scored a goal against Dominick Hasek whom he was traded to Buffalo for in 2001. Attaboy!

It was quite an upset for the Wings, #1 in the NHL, to lose so badly to the Thrashers who rank 21 overall. Or rather, they did until this evening. I'm not sure where this game puts them.

Even worse though is that the Wings have now lost three games in a row. Each game they were trailing by 2 points at some point. It's their biggest losing streak of the season and all of the superstar players are dressed and on the ice. Odd that.

In the end it doesn't matter. Detroit will surely make the playoffs so it was great to see Slava skating at Joe Louis Arena again tonight and leave with what is certain to be a smile on his face.

We miss the old #13, even though the new #13 is definite keeper himself.

Game on!

Killed Two Stones With One Bird

2007-11-28T06:08:00Z

For a long time I've wanted to learn Apple's Cocoa and Objective-C. I've also been very interested in the completely unrelated Amazon Simple Storage Service (S3). Last night I embarked upon a quest to teach myself whatever I could of those by building a rather simple, but cool Ruby on Rails application. What I should have been doing was reviewing a chapter from an upcoming Ruby on Rails book but this was an itch I had to scratch.

Instead of teaching myself Objective-C however I decided to use the wonderful RubyCocoa library. To get at the Amazon S3 service I used the also very cool library that is appropriately namespaced AWS::S3. Removing Objective-C from the equation both hindered and helped. It is a pretty tough language to read so there was a good deal of trial and error when trying to understand some Objective-C code and translate it to Ruby and RubyCocoa.

My inspiration for this tangent comes from Panic, or more specifically their Coda application. It's an awesome tool and I was really impressed by its ability to generate thumbnails of your website or project. Creating website thumbnails isn't new, Alexa at least has been doing it for years. The way Coda does it is inspiring however. They paint nice icons that look like sheets of paper curling up with a well defined shadow, giving it a believable 3D look. It's a small thing, and it's the small things that matter.

My goal isn't to reproduce Coda's thumbnails, I don't have the time or skills to do that yet. My goal is to simply create decent full or partial page screen captures of active websites. The image of the Coda web site here is a result of my efforts. Using a simple web app I enter a URL, the viewport size (default is 1024x768) and the resized version (default is 320x240). If you leave the height off it'll go full height and get entire web page without cropping. Likewise if you leave the resize height off it'll scale it in proportion to the width.

In the end I finally have a good understanding of S3 and I know that I have a long long way to go with Cocoa. Especially if I want to get into Core Image and Quartz.

Here are a few more examples:

My Top 5 Why-oh-Why's of the Day

2007-11-05T22:01:00Z

Just a few questions that I can't seem to find any answers for.

Why do people still use CVS or Subversion when there are tools like Darcs, Mercurial, or Git?
Why are people still developing in and for Windows?
Why are people still declaring variable types ahead of time?
Why isn't Objective-C more popular?
Why did I learn about S-expressions so late in life?

It's been a rough day. I hate trying to figure out how to get a project set up in Visual Studio when all you have are source files.

I'm aggravated at the fact that Subversion is such a piece of crap, but to use anything better you really have to get away from a crappy Windows command prompt and into something more useful, like csh or bash. The droppings left by Subversion in .svn directories (oops, sorry, they are _svn in Windows, WTF?) are super-duper annoying.

I'm also tired of the excessive typing (pun intended) required to develop in C++ or C#. Why can't I just say "x = y" without declaring the type of x; if y is already known. Even worse, why do I have to write code like this (in C#):

XmlDocument x = new XmlDocument();
x.load("file.xml");

foreach (XmlElement e in x.SelectNodes("//foo[@title='bar']") 
{
    doSomethingUsefulWith(e);
}

When all I really need is this (in Ruby):

x = XmlDocument.new
x.load("file.xml");

x.each do |e| 
  doSoemthingUsefulWith(e) 
end

I know, silly arguments. Especially since C# 3.0 brings in Linq and var and a whole slew of other new features to make it more tolerable.

But you know what? You still have to create a bloody class with a public version of the static method main in order to execute. And you still have to wait for 3.0 to come out and stabilize. And then you have to pay for it if you want VS.NET. That's just not right. Yet people continue to pay for it. Suckers? Nah, they just wont listen.

Right on brutha!

2007-10-31T20:38:00Z

Is there any wonder why young people are flocking to this guy? He had it right in '88 and he still has it right today.

Bottom line: Keep it simple you morons. Stop complicating our bloody lives.

Watch and see:

P.S. I just have nothing interesting to talk about in the realm of growing software. So enjoy!

!Off

2007-10-26T02:24:00Z

Sometimes you are just in a really crappy mood.

And you just need something that makes it all make sense again:

Mr. Adams, you are CORRECT sir!

2007-07-06T02:01:00Z

It appears Douglas Adams was right on the money.

According to this article, the weight of life, the universe, and everything (well, almost) is 3x10^42. That's the number three with 42 zeros after it.

Rest in peace my friend.

Fortunately for us the Vogon destructor fleet must've fallen asleep on their way to obliterate the earth for the new hyperspace bypass. Whew.

And now for some Vogon Poetry...

Oh freddled gruntbuggly,
Thy micturations are to me
As plurdled gabbleblotchits
On a lurgid bee.
Groop, I implore thee, my foonting turlingdromes
And hooptiously drangle me
With crinkly bindlewurdles,
Or I will rend thee in the gobberwarts with my blurglecruncheon,
See if I don't!

Ruby on Rails on a CentOS Virtuozzo VPS

2006-12-30T04:23:00Z

I've done this at least a dozen times in VMware, OpenVZ, and Virtuozzo virtual servers. Every time I do it I start from scratch. It's silly. This time I'm writing down the steps so I don't forget.

Getting Started

Let's start with a Virtuozzo powered VPS running a clean installation of CentOS 4.4. We begin by removing a few things that we just don't need right now.

Fetchmail
SWSoft's Apache build
SWSoft's Sendmail build
The Pop3 server that we don't need

Remove what gets in the way

rpm -e fetchmail hspc-wwwroot
rpm -qa | grep httpd | xargs rpm -e
rpm -qa | grep sendmail | xargs rpm -e
rpm -qa|grep ^apr|xargs rpm -e
rpm -e popa3d
service xinetd restart

Install yum

In the virtual server I am on right now, yum is not installed. Odd. Let's install it:

rpm -i http://mirror.centos.org/centos/4.4/os/i386/CentOS/RPMS/yum-2.4.3-1.c4.noarch.rpm \
       http://mirror.centos.org/centos/4.4/os/i386/CentOS/RPMS/python-elementtree-1.2.6-4.2.1.i386.rpm \
       http://mirror.centos.org/centos/4.4/os/i386/CentOS/RPMS/python-sqlite-1.1.7-1.2.i386.rpm \
       http://mirror.centos.org/centos/4.4/os/i386/CentOS/RPMS/sqlite-3.3.3-1.2.i386.rpm \
       http://mirror.centos.org/centos/4.4/os/i386/CentOS/RPMS/sqlite-devel-3.3.3-1.2.i386.rpm \
       http://mirror.centos.org/centos/4.4/os/i386/CentOS/RPMS/python-urlgrabber-2.9.8-2.noarch.rpm

The CentOS-4 Testing repository

Now we want to add the c4-testing yum repository so we will install the latest Ruby rpm's. Or we can install from sources, but no point in doing that right now since c4-testing has Ruby 1.8.5.

cat << EOF > /etc/yum.repos.d/CentOS-Testing.repo
[c4-testing]
name=CentOS-4 Testing
baseurl=http://dev.centos.org/centos/\$releasever/testing/\$basearch/
enabled=1
gpgcheck=1
gpgkey=http://dev.centos.org/centos/RPM-GPG-KEY-CentOS-testing
EOF
yum -y update

Install Ruby, and Rails

As mentioned previously, you can install Ruby from sources or use the RPM provided in the CentOS repository. I use the CentOS built Ruby simply because it's already at the latest version (1.8.5 at this time) and it saves a little time.

Ruby

yum install -y ruby ruby-libs ruby-devel ruby-rdoc ruby-irb ruby-ri

Ruby Gems

wget http://rubyforge.org/frs/download.php/11289/rubygems-0.9.0.tgz
tar zxf rubygems-0.9.0.tgz
cd rubygems-0.9.0
ruby setup.rb

Rails, Mongrel, and Sqlite3-ruby

Rails, mongrel, and sqlite3-ruby:

gem install -y rails
gem install -y mongrel
gem install -y sqlite3-ruby

Apache and Subversion

And now we get to the tougher stuff. We need subversion if we plan to freeze_edge, which I always do during development. I also like to use Trac. We wont cover the installation of Trac today, but we'll make sure our environment is Trac friendly. More on that another time perhaps.

So let's get prepared for Subversion. We'll need the following items installed before we can build it the way we need it:

Apache Portable Runtime (apr, apr-util
The Apache Web Server (httpd)
Neon
SWIG

apr

Starting with apr and apr-util, we need to download, configure, build, and install from sources:

wget http://www.ip97.com/apache.org/apr/apr-1.2.8.tar.bz2
bzip2 -dc apr-1.2.8.tar.bz2 | tar xf -
cd apr-1.2.8
./configure && make && make install

apr-util

wget http://www.ip97.com/apache.org/apr/apr-util-1.2.8.tar.bz2
bzip2 -dc apr-util-1.2.8.tar.bz2 | tar xf -
cd apr-util-1.2.8
./configure --with-berkeley-db --with-apr=/usr/local/apr/bin/apr-1-config && make && make install

Apache Web Server

wget http://apache.downlod.in/httpd/httpd-2.2.3.tar.bz2
bzip2 -dc httpd-2.2.3.tar.bz2 | tar xf -
cd httpd-2.2.3
./configure -C \
            --prefix=/usr/local \
            --with-apr=/usr/local/apr/bin/apr-1-config \
            --with-apr-util=/usr/local/apr/bin/apu-1-config \
            --datadir=/var/www/home \
            --sysconfdir=/etc/httpd/conf \
            --enable-dav \
            --enable-suexec --with-suexec \
            --with-suexec-caller=apache \
            --with-suexec-docroot=/var/www \
            --with-suexec-logfile=/var/log/httpd/suexec.log \
            --with-suexec-bin=%{_sbindir}/suexec \
            --with-suexec-uidmin=500 --with-suexec-gidmin=500 \
            --with-devrandom \
            --enable-cache --enable-disk-cache --enable-mem-cache --enable-file-cache \
            --enable-ssl --with-ssl \
            --enable-deflate --enable-cgid \
            --enable-proxy --enable-proxy-connect \
            --enable-proxy-http --enable-proxy-ftp \
            --enable-proxy-balance && make && make install

Neon

wget http://www.webdav.org/neon/neon-0.25.5.tar.gz
tar zxf neon-0.25.5.tar.gz
cd neon-0.25.5
./configure --with-ssl=openssl && make && make install

SWIG

wget http://prdownloads.sourceforge.net/swig/swig-1.3.31.tar.gz
tar zxf swig-1.3.31.tar.gz
cd swig-1.3.31
./configure && make && make install

Subversion

wget http://subversion.tigris.org/downloads/subversion-1.4.2.tar.bz2
bzip2 -dc subversion-1.4.2.tar.bz2 | tar xf -
cd subversion-1.4.2
./configure --with-apxs=/usr/local/bin/apxs \
            --with-swig=/usr/local \
            --with-ssl \
            --with-neon=/usr/local \
            --with-apr=/usr/local/apr/bin \
            --with-apr-util=/usr/local/apr && make && make install

Good to go

At this point we're good to go. I usually install Vim7 right about now because I prefer to do all my coding in vi. CentOS 4 only has Vim6; Vim7 has newer rhtml and rb syntax files. Adding PostgreSQL or MySQL will complete your stack for a production environment. Sqlite3 is great for getting the ball rolling. Now it's time to be a Really Useful Engine and get working on your shiny new Rails app.

Good luck!

Subversion and those annoying .svn directories.

2006-12-18T03:13:00Z

Nearly a year ago I wrote about a Load-Bearing Kludge. In that article I talked about how we implemented a work-around for getting rid of those pesky .svn directories that Subversion uses for meta-data. The solution wasn't to get around them, but to relocate them. To do this I had to create a shared library that intercepted and decorated a few low-level C library functions like open, fopen, unlink, and a few others.

Since that article last February I've received quite a few requests for the source code that did this. It isn't anything spectacular and in fact the code was abandoned for a much better solution of not using Subversion at all. At any rate, I just received approval from Matt Lightner, the CEO of Site5 to go ahead and give it away. So here it is.

The code

Build it

The provided Makefile is the simplest of Makefiles. It knows how to build the .so, only using gcc. AFAIK this code will not easily work on Windows.

Just copy the fix_dotsvn.c and Makefile into a directory and run make. This should create fix_dotsvn.so.

Use it

fix_dotsvn.so will reroute all ".svn" directory access to whatever directory specified in the environment variable "DOT_SVN_PREFIX", which defaults to "/backup/svn_home". Change it to wherever you would like to reroute your .svn's.

To load it, set your LD_PRELOAD environment variable to /path/to/fix_dotsvn.so before running the svn command. Example: $ LD_PRELOAD=/path/to/fix_dotsvn.so svn update

That's all folks

That's really all there is to it. It worked perfectly while we used it.

Flamethrowers

2006-12-13T19:05:00Z

Think for a moment about the concept of the flamethrower. Okay? The flamethrower. Because we have them. Well, *we* don't have them, the army has them. That's right. We don't have any flamethrowers. I'd say we're fucked if we have to go up against the army, wouldn't you? But we have flamethrowers. And what this indicates to me, it means that at some point, some person said to himself, "Gee, I sure would like to set those people on fire over there. But I'm way to far away to get the job done. If only I had something that would throw flame on them." Well, it might have ended right there, but he mentioned it to his friend. His friend who was good with tools. And about a month later, he was back. "Hey, quite a concept!" WHHOOOOOOOOSSHHH! And of course the army heard about it, and they came around. "We'd like to buy about five hundred-thousand of them please. We have some people we'd like to throw flame on. Give us five hundred thousand and paint them dark brown. We don't want anyone to see them."

- George Carlin, 1984

s/people/code/g

So I'm stuck. I have to fix an application that was born broken. I mean the architecture, the design, the implementation, all of it. Broken. What does work works entirely by accident.

The application is a web application. Written for the ASP.NET platform with C# underpinnings. Not an entirely bad choice of platforms. Until you start looking under the hood. It's a procedural mess wrapped up in pseudo classes that act as nothing more than namespaces. Exceptions are silently absorbed and rarely managed. The so-called data access layer is nothing short of frightening.

The worst part? It's something that I helped build several years ago. I hold my head in shame. Kind of. In fact, several years ago I birthed a framework using ASP.NET and C# that I called "Epicenter." Epicenter was an ultra thin implementation of the Apache Cocoon framework that used XML quite liberally to define application flow and data access and used XSLT for rendering XHTML output. It was neat but it wasn't mature enough to be used in a production environment. Sso the choice was made to use the standard ASP.NET WebForms. I got frustrated and have long since lost the code for Epicenter. If this were on fark, I'd clearly deserve the Dumbass tag.

All I can say about this today is: Ooops, where's that bloody flamethrower?

Personal Projects

2006-11-02T03:00:00Z

I always seem to be meddling with one thing or another. More often than not I end up throwing away whatever it is I am working on because, well, I'm not all that interested in the final product. I'm more interested in the intellectual challenge or learning something. Occasionally though I really do build something useful, and then I end up throwing it away for no good reason at all.

It's time to stop doing that. Start keeping my projects online so they never get totally lost. So look up. There's a new Projects tab up there. Watch it. See what I've done or am doing. It could be boring, or it could be fun. It could be useful or it could be totally useless. Who knows? You certainly wont unless you watch.