The Tale of HitTail Development

by Mike Levin SEO & Datamaster, 07/02/2006

So, I made the somewhat controversial decision to develop the HitTail application in ASP Classic–very out of vogue these days. There are so many other programming choices, but all of our programmers at Connors have Visual Basic Scripting Edition (VBScript) programming experience, as well as myself. And I wanted to develop fast and have freedom in distributing the work load. We developed HitTail with rapid porting to other languages and database subsystems in mind, and deliberated hard before choosing Microsoft–and older Microsoft technology at that.

I had some very precise requirements. First, we had to be in familiar territory and couldn’t hit unexpected “gotcha’s”. I had to be confident that when we hit it big, we could scale fast. Switching to new platforms, especially ones as un-tested as Ruby, as appealing as it was, was just a little too unsettling with an app like HitTail, where the system would be invoked for every page load of every page of a site that’s using it. And none of us are Java-heads.

In fact, of the two major enterprise development platforms out there, J2EE and .NET, I spent considerable time attempting to “move over” to each. My early attempts at Java circa 1999 are well documented on the Internet, and are still often used as “Hello World” reference. Java just defeated me. I am a part-time developer, and if you’re not ready to commit yourself heart and soul to a language, Java doesn’t seem like the right choice. I tried learning C++ once, and quickly gave up. The syntax confused me, and I got the same C++ feeling from Java, but without the dangerous memory pointer issues.

Now, I wasn’t a green programmer. By this time I already knew PERL, TCL and VBScript. So, it was with this Web-programmer mentality that years later I exuberantly embraced .NET, and ASP.NET… only to be stung again. The same thing that turned out being the biggest assets of these enterprise languages was my downfall–a system class library as large as an encyclopedia and methodology that I didn’t necessarily agree with. Everything became a scavenger hunt just to achieve the littlest things. And if you didn’t want to use the built-in methodology, you gave up what the environment had to offer, and might as well be using a language you like more in the first place. My boss at the time (circa 2002) had me doing very Ajax-like things (before Ajax), and I fought .NET every step of the way. Further, all the ASP.NET goodies like the DataGrid were philosophically and methodically incompatible with the app we were trying to build. This is also very well documented on the Internet, and continues to be broadly referred to by websurfers for tabs and iframe control under .NET. Bottom line is that my exuberance for .NET quickly turned into frustration.

I kept defaulting back to VBScript, and even built 3 generations of a rapid agile development system under VBScript. Again, it was Ruby on Rails before Ruby on Rails. With it, I developed social networking software that was exclusively used by one company for sales lead management and order management purposes. It was like blogging and MySpace before Blogger and MySpace. I seem to have anticipated every trend and made a “junior” version of it in-house before some more fully deployed and marketed version of it exploded onto the scene.

Well, not again. I’ve been writing for the long tail of search since about 1999, starting perhaps with Prophet 21 (now Activant), where I had to figure out what people researching business systems for the wholesale distribution industry would be searching on. I created a killer search-optimized content management system that could tie in with any database or XML source and spin out sequences of perfectly optimized HTML pages using the “formula du jour”. It was done under VBScript, but this is when I was learning Java, and experimented with parts of it under Java. Here we are in 2006, and I’m working for the PR company that launched Amazon and Priceline. And I’m sitting on top of the next big thing that’s ready to bust out mainstream in a big way. And I’m not going to let anyone eat my lunch this time.

And so it was in this mindset that I chose VBScript for development. Controversial, yes. Doable, yes. Pitfalls, none. The only pitfalls are those of rhetoric and dogma, which I’ve already been encountering. When you look at the performance specs of IIS in the areas that are important for an application like HitTail, such as simultaneous connections, you come up with some very favorable numbers. It won’t take a lot of IIS servers to service the initial surging demand for HitTailing. And the tired old security arguments are moot, because security is more a matter of configuration than inherent vulnerabilities. It does what I need it to do: serve vast quantities of static HTML at backbreaking rates over relatively few servers.

And that brings us to the SQL Server decision. Yes, MySQL is quite popular, and is embraced by companies looking to dodge a licensing fee. But if you’re going for the absolute best transactions per second performance and minimized risk of data loss, go researching and tell me what you find. Real apples-to-apples benchmarking between SQL Server and MySQL are few and far between, and when you do find them are not very confidence-inducing. Compare this to the war that Microsoft has been waging in the database space to catch up with Oracle and IBM DB2. It is much to their chagrin that the credible industry benchmarking by objective organizations shows Microsoft SQL Server outperforming the others in small database conditions time and time again. Small databases these days are under 3000 GB. So at the price, SQL Server is a very good deal, with rather astounding performance characteristics, and is even safeguarded against sudden power loss. MySQL isn’t safe against power loss, and isn’t exactly free either. So, SQL Server is a fine choice for an app like HitTail.

The next thing to keep in mind is that we’ve programmed HitTail so that all the communication between components takes place with Web Services. Therefore, we are not really closed into any particular technology, and can swap out any underlying pieces with the best of breed system, regardless of price, when the app starts to take off. The static HTML can be moved to a content delivery network like Akami or Panther. The tracking gif can be moved to Linux Apache server farms. The database can be moved to IBM DB2. Indeed, we have planned HitTail to scale to massive proportions. Yet, we were able to jumpstart it using the most readily available skills and technologies.

Now, I love Ruby on Rails and the whole concept of agile frameworks. I love scripting language environments whose code becomes compiled in memory for re-execution, like Active PERL and Active Server Pages. I’m a little bit less of a fan of being locked into a particular programming language that must run on its own virtual machine, like Java (JVM). .NET is a bit better, because even though it has it’s own equivalent of a virtual machine in the common runtime library (CRL), you can use the programming language of your choice. But I think scripting languages with runtime compiling and persistence running directly on the underlying systems without a VM are best–especially when they have equal implementations on multiple platforms. PHP, Python, PERL, and yes–VBScript are such examples. RUBY is shaping up to be one as well. And Java can be used this way with JSP.

But these languages are commonly criticized by the snobbish enterprise programming community (Java in particular) for not being scalable or having some other intangible asset that “enterprise” languages allegedly have. But I think that scalability more a matter of programming techniques than inherent characteristics of a language. In fact, I’d go further to say that appropriateness for enterprise use (thousands of simultaneous users per second) is a matter of the framework you put between the programming language and the application.

In fact, the rapid ability to refine an application in response to what you learn from users is a more important characteristic of enterprise-class software than a class library the size of the Aristotle’s full writings. In this sense, Ruby on Rails is better for enterprise development than Java Struts. By the time a Java person has gone through a product spec, a Ruby developer could be on the third iteration of a spiral development cycle. Then, the only thing ensuring very large scale use is stability and execution speed–things which are falling into place as we speak. Ruby on Rails has the extra double-whammy of tweaking Java people by being a true and pure object oriented environment in a scripting environment whose syntax is more concise and beautiful than Java… by far!

It is in this spirit that I eventually plan on re-coupling the HitTailing system back end user controls to either next-generation Ajax on Rails, or even Flash Flex Server with ActionScript. Bottom line is that even though we used VBScript today for a rapid and agile spiral development cycle, we can use the coolest, best, and most forward-thinking technologies for each sub-system as we build HitTailing up into something as mainstream and well known as AdWords.