The Miseducation of S. Potter

Written on 11:57:00 AM by S. Potter

Ok, obviously the title was inspired from Lauryn Hill's first solo album, but the subject at hand is the essence and natural language of solving problems. If any of you have been reading the last few entries at "Snakes, Gems & Coffee" you would know that I have been very interested in linguistics ideas as they apply to programming or domain-specific languages. Over the last two weeks I have been involved in a new project. Previously I had been consulting at a firm that provides hosted software solutions for asset management firms. The software platform that we used solves well known and fairly well understood problems in that industry. Some of those problems are mathematical, or at a minimum computationally numerical in nature. For example, there are industry standard and widely accepted formulas that calculate future and present values for applicable financial instruments and cash flows. Of course, the values you use for some of the variables in each formula can be tweaked or disputed, but overall you have the plan, the framework to get to your end goal. Now on my new and current project I am faced not with coding a solution to a well defined or well understood problem, but actually defining the problem space from scratch. Since I am a Mathematics graduate I thought this should take me right back to my college days, then it hit me; I learned more about defining problem spaces in high school calculus than I did in college multivariate calculus equipped with my partial differential equations, proofs and Taylor series'. The problem in my college education in Mathematics was two fold: (a) the problem space was already defined in a domain specific vocabulary of pure mathematics; and (b) since the problem space was far more higher level than my previous math education, we lost touch with real world applications. Solving well defined problems is only one half of problem solving as far as I am concerned and a degree in Mathematics from a university that is highly regarded in the field did not teach both. In high school I read books about Euclid's The Elements and Newton's findings. These books described how Euclid, Newton and other famous mathematicians observed the real world to inspire their mathematical findings. I was under the impression that a Mathematics education at university in the 20th century (yes, I am old, I graduated in the previous century) would also not loose touch with the real world. I found the opposite. I agree that proofs and pure mathematics are an extremely important discipline to master, but I felt robbed when I realized that was the extent of what the department offered me, including the "applied" mathematics courses the department offered. Today I feel increasingly this way about software product management. Since college I have been a software developer/engineer in one form or another, from project peon to project manager. In that time I have been learning natural ways of solving problems in the real world and over the years have realized that one of the biggest problems in our industry, one we need to work on more, is defining the problem space. Obviously there are ways to manage around vague or inaccurate software requirements such as employing agile methods to describe, estimate and re-evaluate in-depth user stories directly with the customer or end-user, but that is only managing the risk and process. What is needed is defining not just the problem space, but the methods and language(s) we need to converse in about software requirements, user stories, use cases, functionality, features or how ever else you wish to dissect the same thing. Let's look to our wise ancestors like Euclid and Newton to draw inspiration not from textbooks, but by observing the real world around us. The evolution of software products/projects toward agile processes is a step in the right direction, but let us not loose sight of the real world. I've met a few good developers that read all about agile and talk a good talk, but don't practice what they say. Let's all get back to basics and let the real world inspire us equipped with our textbook theory and engineering discipline. Happy holidays and look forward to your thoughts on this or related topics.

Agile Holiday Reading

Written on 9:53:00 AM by S. Potter

Over the weekend a friend who is just starting to head up a new web product team asked me for for some pointers on good agile project/methods books. I'll try to make my choices as technology agnostic as possible, but some of the material might be more specific to my friend's technology stack (Java/Spring/Hibernate). Not all of these are new books from 2006, so this is definitely for the project manager or technical lead that hasn't read many books on agile software. Agile and Iterative Development: A Manager's Guide
by Craig Larman - Addison-Wesley Professional; 1st edition (August 15, 2003)

Books written for technical managers tend to just fluff over a large portion of the practical nature of managing software projects. This is different.
Test Driven Development: By Example (Addison-Wesley Signature Series)
by Kent Beck - Addison-Wesley Professional; 1st edition (November 8, 2002)
Good first read for those just starting TDD. Not for the TDD guru!
Refactoring: Improving the Design of Existing Code
by Fowler, Beck, Brant, Opdyke, Roberts - Addison-Wesley Professional; 1st edition (June 28, 1999)
Great book whether the project already has a large existing code base or if you are just starting out, refactoring is almost the most important part of agile software projects. As long as your remember that without the test assets agile software products should possess, refactoring would be (and is for many non-agile developers) highly error prone. Remember the refactoring strategies outlined in this book heavily rely on being test-infected on your project. Don't try this at home without having a solid arsenal of unit, functional and integration tests for your product.
Agile Java(TM): Crafting Code with Test-Driven Development (Robert C. Martin Series)
by Jeff Langr - Prentice Hall PTR (February 14, 2005)
Good for those needing to learn JDK 1.5 with TDD best practices specific to Java.
If you have favorites that provide different insights or focus on different aspects of agile software methods, please send us the information via comments.

Beautiful Code: Domain Specific Languages

Written on 5:10:00 PM by S. Potter

Last year when I was first introduced to Ruby on Rails through the screencasts, I remember watching DHH (Rails creator David H. Hanson) say that one of the main motivations behind developing Rails for him was to allow him to create "beautiful code" and that writing beautiful code made him happy and more productive. At the time I liked the concept, but wasn't really sure how true that would be, at least for me. I am a non-visual person and I really didn't think writing beautiful code would affect the way I work and make me noticably more productive. Well, I was wrong! Since last year (2005) when I started Rails development I have grown very attached to creating beautiful code, so much so that have noticed a remarkable difference in how productive I am in Ruby and Ruby on Rails projects compared to my Java work. Remember in 2003 I thought Java was the dog's tuxedo and only ever wanted to write Java (oh, how things have changed). Late last night I watched a Ruby SIG talk about designing domain-specific languages (DSLs), which peaked my interest in the relationship between Sapir-Whorf, beautiful code and developer productivity. Of course, I heard about Sapir-Whorf a while ago and found it interesting at the time, but wasn't really applicable to what I was working on at the time (or at least I thought), so I didn't pay much attention to it until now when almost every Ruby presentation discusses it. Until this evening, I was not able to work on CodeGenie since my last blog entry, but I think it was for the best. My new found love for rspec (a DSL for spec-ing software written in Ruby) and my interest in the relationship above both have inspired me to think about CodeGenie in a new light. Let's take a step back. The general idea behind the Sapir-Whorf hypothesis is that the language you use to "discuss" a subject affects your internal thought processes, such that using one language could restrict or expand your thought process compared to another language. This is applied to both natural and programming languages (in programming languages I include both DSLs and general purpose programming languages that must be Turing-complete, e.g. Ruby, Python, Java, C++, COBOL, etc.). For those that haven't looked at rspec, here is a quick snippet of code:

context "An empty Array" do
  setup do
    @array = []
  end
  
  specify "should be empty" do
    @array.should.be.empty
  end

  specify "should have size of 0" do
    @array.size.should.equal 0
  end
end
Of course, this is pretty simple stuff, but you can immediately read what the snippet specs out for the Array class. It uses language the is specific to the domain of specifying software requirements. In a loose way Rails provides DSLs for defining controller actions, data schema migrations, object-relational mapping, web services and whatever else you use the Rails environment for. However, rspec really goes the extra distance in defining a "language" to use for the sole purpose of specifying software requirements. In many ways this supercedes Test::Unit (or JUnit for the Java heads out there) - you should check out Dave Astels' presentation at Google on Behavior Driven Development (BDD) [very enlightening]. In CodeGenie I want to create a DSL that allows plugin developers to define generators and/or metamodels implemented within the plugin. The idea is that a plugin can contain one or more generators or metamodel implementations. Each generator defines one manifest per metamodel that it supports. A metamodel will define a translator that sucks in the metamodel data, parses it and then spits it out in an in-memory object representation custom to the metamodel format. CodeGenie itself will provide a set of very standard generators and metamodel implementations. Currently am thinking the plugin developer will define a plugin with one generator, something like the following snippet:
context = CodeGenie::Context.new(:TEMPLATE_DIR => "templates", :DOCUMENT_DIR => "documents")

generator "Rails startup generator" do
  manifest :yaml, context do
    # generate setup.py using data from the YAML metamodel passed in to the eRB template
    template "#{TEMPLATE_DIR}/setup.py" "setup.py"
    # create src directory in the root directory of the destination path.
    dir "src"
    # copy standard README, LICENSE and INSTALL files without processing it through the eRB engine - straight copy
    file "#{DOCUMENT_DIR}/README" "README"
    file "#{DOCUMENT_DIR}/LICENSE" "LICENSE"
    file "#{DOCUMENT_DIR}/INSTALL" "INSTALL"
  end
end
For those familiar with the Rails generator, this should like slightly familiar, except that instead of extending a Rails class and defining a manifest class method, I have created a bit more syntactic sugar to make it more domain-specific. These are my initial thoughts and I will work on maturing the DSL over the next couple of weeks. My plan is to get a simple, but working CodeGenie released with YAML and limited MOF support by the end of 2006. Thanks for your patience and if you have suggestions for CodeGenie, please send a message to codegenie-suggest@rubyforge.org

Sapir-Whorf, Domain Specific Languages & Gay Marriage

Written on 6:46:00 PM by S. Potter

Sapir-Whorf

A while ago I was introduced to Sapir-Whorf, which is an intriguing linguistics hypothesis. Informally (my definition) it states that the language you use to describe a subject shapes the way you think about that subject. Formally Wikipedia defines the Sapir-Whorf hypothesis as:
there is a systematic relationship between the grammatical categories of the language a person speaks and how that person both understands the world and behaves in it. This controversial hypothesis is named after the linguist and anthropologist Edward Sapir and his colleague and student Benjamin Whorf.
Almost everywhere I look these days, especially in the domain specific language design discussions, Sapir-Whorf is cited.

Domain Specific Languages

A domain specific language (DSL) is a programming language which may or may not be Turing complete that was design for the purpose of executing specific tasks. Examples of DSLs include:
  • rspec - a software requirements specification DSL
  • Maven POM - a project comprehension tool's DSL
  • awk - processes text data DSL
  • XSLT - XML transformation DSL

Gay Marriage

That all sounds very academic, so I wanted to take a few steps back to have a look at a real world example and extension of Sapir-Whorf, especially applicable to the US audience today. Until 2004 I do not believe the majority of voters in the US had considered what "gay marriage" actually meant. During the 2004 presidential elections it was at the forefront of the social issues in the debates even though both major presidential candidates (George W. Bush and John Kerry) were both against "gay marriage". In the public and media debate on "gay marriage" the social right characterized "gay marriage" as breaking down the "institution" of marriage. On the GLB and liberal side people characterized "gay marriage" as one step in a broader campaign promoting equality for a minority group analogous to the civil rights plight in the 1950s through 1970s but for the GLB minority. Today "gay marriage" remains a hot button issue for many Americans as we head into the mid-term elections.

Sapir-Whorf & Gay Marriage

My personal (religious/political) opinion on the matter of "gay marriage" is irrelevant to my application and extension of the Sapir-Whorf hypothesis in the discussion below. To understand my lay person's (note: I am anything, but an expert on this subject) linguistics argument you should also try to keep your personal opinions out of your mind when you process the discussion. According to Wikipedia:
A marriage is a relationship between or among [two] individuals, usually recognized by civil authority and/or bound by the religious beliefs of the participants.
Let us investigate what the GLB and liberal groups advocating marriage are referring to. Many references to gay marriage on the HRC (Human Rights Campaign, which advocates "gay marriage") website use the terms "gay marriage" and "civil unions" interchangeably and the meaning of civil unions seems pretty clear - a set of legal rights between individuals that enter into the union.

The Polls

The Washington Post and ABC News conducted a poll of 1,202 randomly selected adults in US in March 2006 found the following:
  • 59 percent of respondents oppose gay marriage (up 4 percentage points from February)
  • 51 percent of respondents favor allowing same-sex couples to enter into civil unions with the same basic legal rights as married couples (up 6 percentage points in less than a month)
This puzzles me: how can 59 percent (and rising) respondents oppose gay marriage, and yet also 51 percent (and rising) from the same respondents also favor same-sex civil unions with all of the same legal rights as heterosexual married couples? Even considering the margin of error, in this case 3%, this poll strongly suggests that many Americans (assuming the poll respondents are representative of the general population) are not interpreting "gay marriage" and "civil unions" to be interchangeable as GLB groups that advocate "gay marriage" do. Now let us consider the demographic of the social conservatives that vehemently oppose "gay marriage": people that consider themselves religiously active. People in this demographic would probably use the term marriage in a way that emphasizes the religious sacrament of marriage rather than a marriage simply for legal convenience. Of course, people get married everyday for a combination of these reasons - sometimes even business reasons, but as you will see it does not affect the rest of my argument. In fact, the social conservatives' argument against "gay marriage" demonstrates their different interpretation of the term "gay marriage". I suspect social conservatives feel like their freedom of religion is being attacked, but this direction in the argument is long and windy.

Convergence

My suggestion then on the issue of "gay marriage" in the US is simple: the social conservatives and the advocates of "gay marriage" are not talking about the same thing at all AND they are NOT talking in the same language. Whether you agree with allowing same-sex couples to enter into a relationship recognized for legal purposes or not, let us all agree that there is a mismatch in meaning between what GLB groups are promoting as "gay marriage" and what the social conservatives who oppose it denounce as "gay marriage". I also suggest that the reason why these two groups on either side of the debate are talking about different things is because the dialects that they each use, while both are American-English in a broad sense, the meaning of words have subtle differences due to cultural nuances such that on this issue cause a potentially unnecessary collision of large proportions. Below is a proposal for a domain specific language for the purpose of defining political debates. It would look something like this for the issues of Gay Marriage:
political_debate "Gay Marriage" do
  argument_for do
    define :glb_people.includes :gay, :lesbian, :bisexual
    define :civil_union, legal_contract.between.2.individuals
    define :gay_marriage, civil_union.for.glb_people
  end

  argument_against do
    define :glb_people.includes :gay, :lesbian, :bisexual
    define :sacrament, :religious_contract.with.god
    define :gay_marriage, sacrament.for.glb_people
    candidate_affiliation :george_w_bush, :john_kerry
  end
end
Immediately from the snippet of code using my fictitious Political Debate DSL (PDDSL) that the primary for and against arguments have different definitions and that the debate needs to be restructured so that both sides are actually talking about the same thing. My suggestion would be to call it a debate on "civil unions" and NOT "gay marriage". I think this would end a lot of unncessary disputes on the political air- and blog-waves.

OSS: Code Generation for the Rails Generation

Written on 7:31:00 PM by S. Potter

As a Rails developer I have become very accustomed to and depend on it's code generator and third party generators that are available to automate development tasks. I like the way the Rails generator is designed, however, it is highly coupled with the Rails framework, so if you want to use the generator for other Ruby (or other language) projects that are not web services/web applications, then it isn't very useful. Another area that the Rails generator has room for improvement is to support metamodels to pass into generator templates. Today I felt the need to create a Rails generator-compliant code generation toolkit into my own OSS project that can be used in any Ruby, Java, C++, Python or other projects that are not pinned to the Rails environment. Today CodeGenie was born! I have started working on the first set of user stories, which includes the following:

  • Run Developer: execute script that generates code as per plugin and metamodel passed in as arguments.
  • Run Developer: execute script that removes previously generated code files as per plugin and metamodel passed in as arguments
  • Run Developer: execute script that updates previously generated code files as per plugin and metamodel passed in as arguments
  • Plugin Developer: Rails generator compliance [Rails generator drop-ins]
The initial metamodel format I will support in CodeGenie will be YAML. After that I will focus on working on more standards based metamodel formats like MOF, UML and XML Schema. Three years ago I worked on a very heavy model-driven architecture (MDA) project where we determined around 95-98% of the [C++] code was generated from a UML metamodel. Since nobody really likes writing boilerplate C++ code, we wrote a code generation toolkit in Python that read in MOF, UML and XML schema metamodel representations and passed in-memory metamodels (built from those formats) to our code templates, which produced the vast majority of our server logic (C++ and Python), server/client stubs + client APIs (in Java, C++, Python, C#) and .NET user-interface forms in C#. At the time I had some (and still do) major grievances about the way we bastardized Rational Rose 2002 MDL file with UML properties to attempt to support the 33% of the UML 1.4 standard that Rational seemed to have forgotten about when writing Rose 2002 and 2003. This is more of a tool issue rather than the way we did code generation, so I will not worry about this in the development of CodeGenie as common sense should dictate to you what metamodel format(s) and CASE tool(s) make sense for your development environment and CodeGenie plugins. My experiences above contribute to why I want to make CodeGenie completely independent of CASE tools and only provide support for standards-based formats. I don't consider Rose's MDL format standards-based (sorry Dean).

Beck on Smalltalk vs. Java vs. Ruby

Written on 3:50:00 PM by S. Potter

Here's a great quote I found the other day that I wanted to remember. I also wanted to put it out there to get some Java heads to respond:

I always thought Smalltalk would beat Java, I just didn't know it would be called 'Ruby' when it did. -- Kent Beck.

Configure This!

Written on 3:45:00 PM by S. Potter

Lately I have had the severe misfortune to return to the J2EE/JEE5 arena after a blissful haitus of 9 months in pure Ruby (and a little bit of Python, for good measure). I have come to the conclusion in a previous life I must have been a serial killer and now karma has come back to bite me in the ..... In theory, being able to configure every detail of the whole world around you sounds great. Afterall, flexible software is a key selling point for management to buy-in to your project. But at what point did we decide to sacrifice developer productivity and sanity for redundant flexibility? Sure JEE5 folks will say: "What configuration? We got rid of all those ugly XML files with Annotations, so you got no'hing on us". I say: "If a bird floats on water, quacks and waddles, it is, most likely, a duck!". If we only take away one thing that Rails demonstrates well, it should be how conventions should be applied over mandatory configuration for the majority use cases. Java heads take note or Java might as well be dead in the water. With .NET and more agile languages like Ruby and Python as competitors for real world server, utility and UI applications, what hope does Java have of surviving the future? Sure, just like COBOL programmers, Java heads will be needed for legacy system support for years to come, but when good, seasoned and well-rounded software engineers are asked what they really need from a development environment, they overwhelmingly shout from the top of their lungs, "productivity and sanity!" At every turn in J2EE we need to read, write or respond to properties set by either the container, the -D parameters passed in and sometimes environment variables set by the deployer. For example, we can't access another EJB (even locally) without grabbing a set of properties that defines our "context". Only then can we have access to the EJB and invoke methods on it. I am sure the Java heads out there are thinking:

What else would someone do? We need all that flexibility in case we change the JNDI implementation, the EJB container, the database, etc.
Apart from giving them the easy response of "How often to production systems swap out core infrastructural peices like that?", I will suggest Java heads consider the following, "How many times has a J2EE/JEE5 developer swapped out an infrastructural peice in 5 minutes with all this configurable flexibility?". Sure we can do a lot through annotations now and that does improve code readability and declutter somewhat, but doesn't solve the real issue. I suggest that perhaps the Java world has been chasing a dream that does not mesh with reality and one that has severly sacrificed our productivity and sanity.

Rubyisms: reopening classes

Written on 2:26:00 PM by S. Potter

Ever wished you could write code in a very expressive way like the following examples?

  • expiration_date = 3.years.from_now
  • birth_date = 32.years.ago
Now you can, in Ruby at least. Two years ago I was a huge Python head that thought no other language could compete with Python for developer productivity. My Python indoctrination occurred after 7 solid years of proclaiming Java was the ultimate language that would reign supreme. Prior to my Java phase I was a C++ and Perl coder. It was also two years ago that I got my first glimpse at Ruby. I read articles, blogs and forums between Python and Ruby heads discussing finer points of each language and how one is far more superior than the other, because....bla bla bla. It was at this time that Ruby's concept of reopening classes came to my attention. As a hearty Python developer at the time I came to the conclusion that Python had to be superior on this point because Python allowed class reopening through metaclasses, so the original author of the code could be explicit to allow class reopening or not. In Ruby, however, anyone could reopen your classes, write malicious code and redploy without others knowing. Over the past two years I have realized that while Python is a beautiful programming language (not just scripting language) in its own right, my views on this point have changed considerably. To show class reopening I will be using an example of code that is very similar to some Ruby standard library extensions found in Active Support included in Rails (Disclaimer: This is not necessarily the same underlying code - I have not checked to be honest). Let us start up an irb shell session and write the following:
  irb> 3.class
  => Fixnum
  irb> class Fixnum
  irb>   def years
  irb>     return self*365*24*60*60
  irb>   end
  irb> end
  => nil
  irb> 3.years
  => 94608000
  irb> 3.years.class
  => Fixnum
  irb> class Fixnum
  irb>   def ago
  irb>     return Time.now() - self
  irb>   end
  irb>   def from_now
  irb>     return Time.now() + self
  irb>   end
  irb> end
  => nil
  irb> 3.years.from_now
  => Sun Aug 09 14:23:10 CDT 2009
  irb> 3.years.ago
  => Mon Aug 11 14:23:19 CDT 2003
Did you follow what I just did? I wanted to show what 3's class is in Ruby. To those not very familiar with Ruby, this was important to show, since newbies might expect 3.class to be Integer, but in fact it is of type Fixnum. Next we reopened the Fixnum class for new definition. If we had reopened it and inserted a method called floor or ceil we would have overwritten the original Ruby standard library implementation of these methods. Instead we defined a method called years, which wasn't previously defined in Fixnum. Then we closed the class and found that 3.years returns 94608000, which is the number of seconds in a year. Is that useful? Not yet, so next we reopened Fixnum again and insert two methods: ago and from_now. After reclosing the class again, we can write partial sentences in Ruby and provide an extremely expressive API for an application that deals a lot with dates and converting numbers to dates. This couldn't be done in Python with the same ease using metaclasses, because the builtin int type cannot be reopened unless you redefined the int type from scratch in Python including the addition of your special purpose methods in the mix. Anyway, this might not convince all readers of the significant utility of the class reopening feature of Ruby, but I have began to love this feature, arguably too much (some may say). I have found it has significantly improved readability and condensed the code necessary to do many tasks. After converting to Java a number of years ago from Perl and C++, I vowed not to take the Perlist's attitude toward trusting clients of your code not to bastardize it. However, I find myself accepting that if a client of my code knowingly decides to take this risk, it is on their own head they fall. I should not be bothered by this professionally or personally. I should also bring to reader's attention that this feature of Ruby is a stepping stone to promote the development of true domain specific languages (DSL), which is difficult at best to do in many other languages.

Rubyisms: naming conventions

Written on 12:43:00 PM by S. Potter

Today I wanted to show the implicit and explicit naming conventions in Ruby for newbies. It isn't particularly consistent with another language's conventions, so could cause confusion when reading the Ruby Standard Library (RSL). Where appropriate I also discuss how some of the naming conventions really aid code readability and developer productivity.

ClassName and ModuleName [implicit]

Naming classes is very similar to most common conventions following in C++, Java, Python and C#. e.g. DataMigration, CorporateConvertibleBond, Controller etc. I am unaware of a class name in the (RSL) that violates this rule. RSL examples of convention usage are Fixnum, Rational, DateTime, Module, NameError, NotImplementedError, NilClass. I realize Fixnum is a little controversial, but I think there is a case for it's naming as above.

variable_name and regular_method_name [implicit]

For variable names and methods that do something that isn't dangerous and/or return a non-boolean to the caller, the common underscoring convention is used to name them. This is consistent with Perl and PHP variable naming conventions. RSL examples include: String#strip, Object#instance_eval, Array#sort_by. Please note this is the one convention listed here that is violated in the RSL the most.

method_asking_a_question? [implicit]

Suffixing a method that asks a question with a '?' is a great way to tell clients of your code it returns a boolean value to be used in conditional expressions. My favorite RSL example is Object#nil?. This is one of the most important methods, in my mind, that exist in Ruby in conjunction with making Ruby's None, null, NULL equivalent, nil and object of type NilClass. RSL exaples include: Object#is_a?, Hash#has_key?, Object#frozen?. This naming approach is my joint favorite with the following convention that significantly increases readability of code and implicitly improves developer productivity as a side effect.

slightly_dangerous_method! [implicit]

For methods that change the internal state of the object in a way that may not be apparent, a '!' is suffixed to the method name to follow this convention. As an example in the RSL, you would probably expect the method "a string".upcase to return a copy of the original string in upper case, but the method "a string".upcase! would probably warn you that it would change the original object's internal state. This is an important distinction to know as a developer writing client code to the String class. This naming convention will keep you out of trouble. If a team mate decides to try out some of your APIs, then you can feel safe knowing you have warned him/her of potential danger spots.

@instance_variable [explicit]

Instance variables are explicitly prefixed with the '@' sign. Some may thing this is a Perlism that crept over, but in Perl '@' prefixed variables referred to arrays, if I remember correctly, so the usage is quite different. It should be noted that the '@' prefix is only used when referring to instance variables internally within the class. Using the '@' prefix does provide a convenient visual que that I am using an instance method which definitely improves readability of code.

$global_variable [explicit]

To refer to global variables prefix the name with '$'. This is an explicit naming convention in Ruby, much like the '@' prefix for instance variables above. This gives a quick visual note that we are knowingly using globals. For me it also provides a psychological check, where I start to question my motives whenever using a global variable and usually end up refactoring code to avoid using globals.

SOME_CONSTANT or OtherConstant [kind of explicit:)]

Much like in Java and Python, Ruby's constant naming convention is not commonly consistent. Often developers use the capilized naming approach, while just as many use the camel case convention used for class and module names in Ruby. Personally, if the constant is accessible from the module or class, I default to using the former approach unless the project has a different preference. For internally accessible constants, it is probably a moot point. All in all I find using these or very close naming conventions to those listed here, help me become more productive as a Ruby developer. I also find it useful to notice the Ruby mindset of being as expressive as possible without being verbose implicitly defined in the naming rules above.

Rubyisms: forwardables

Written on 6:23:00 PM by S. Potter

Recently some Java friends of mine have decided to taste the juicier fruits in Ruby-land with my assistance. So below are some excerpts from an email conversation I had with one about Ruby's standard library forwardable features:

Suppose we have the following model classes defined for a simple CRM system: Customer, Address, PhoneNumber, Name, etc. Now in Java-land we would have written something that looks like the following Ruby code (except you must type about a hundred more lines - although now you can write annotations to automate basic things like generate getters and setters or the like, which still requires a ridiculous amount of code to accomplish):


Name = Struct.new(:first_name, :last_name, :suffix, :salutation)
Address = Struct.new(:street_address, :suite, :city, :state, :country, :zip)
PhoneNumber = Struct.new(:country_code, :area_code, :exchange, :number, :extension)

class Customer
   attr_accessor :name, :address, :phone_number
end

Note: the attr_accessor will create getters and setters for Ruby newbies.


The problem is that you are not sure when you first design this if it will support future product offerings for business customers, which means the way we have modeled our simple CRM above may not support a business customer as well as individual customers. However, as good agile developers we know better than to pre-empt future requirements by modeling the world up-front, so we stick with our current implementation for this next iteration/release. To help us with future changes (even though we do not know what they will be or try to pre-empt them) we want to be able to access details directly from the Customer object to reduce coupling of clients of our Customer model. This also helps us implicitly adhere to the Law of Demeter (when not misused). A naive approach would be to manually create accessor methods for the Address, Name and PhoneNumber attributes in the Customer class. But remember, this is Ruby-land, which is a magical place, so we can just do the following:


require 'forwardable'
NAME_FIELDS = [:first_name, :last_name, :suffix, :salutation]
ADDRESS_FIELDS = [:street_address, :suite, :city, :state, :country, :zip]
PHONE_NUMBER_FIELDS = [:country_code, :area_code, :exchange, :number, :extension]
Name = Struct.new(*NAME_FIELDS)
Address = Struct.new(*ADDRESS_FIELDS)
PhoneNumber = Struct.new(*PHONE_NUMBER_FIELDS)

class Customer
   extend Forwardable
   attr_accessors :name, :address, :phone_number
   def_delegators :@name, *NAME_FIELDS
   def_delegators :@address, *ADDRESS_FIELDS
   def_delegators :@phone_number, *PHONE_NUMBER_FIELDS
   def initialize(name, address, phone_number)
     @name, @address, @phone_number = name, address, phone_number
   end
end

One of the many fantastic things about Ruby (and Python) is we have irb Ruby's interactive shell to prototype with. So we open up a irb shell and do the following:



 irb> require 'FILE_WITH_ABOVE_CODE_IN'
 irb> john_adams = Customer.new(Name.new('John', 'Adams', nil, 'Former President'), Address.new('101 Constabulary Road', nil, 'Bay Colony', 'MA', 'US', '000000'), nil) # remember phones didn't exist in the mid-18th Century! => #<Customer:0xb7f2be30 @address=#<struct Address street_address="101 Constabulary Road", suite=nil, city="Bay Colony", state="MA", country="US", zip="000000">, @name=#<struct Name first_name="John", last_name="Adams", suffix=nil, salutation="Former President">, @phone_number=nil> 
 irb> john_adams.salutation => "Former President"
 irb> john_adams.state => "MA"
 irb> john_adams.street_address => "101 Constabulary Road"

Javahead>>> We are still passing in the Name, Address and PhoneNumber stucts to create the Customer object in the first place, so why would flattening out the Customer interface reduce coupling?

Great question [Javahead]! In general, most applications will read data from models much more than create them, and quite a number of applications do not actually initially create any of the data, they simply massage it. Now I am not suggesting that there are no applications that would create model objects more than read from them, but most likely your application will be 80% reading and 20% writing (including creating and updating). So if we were to change the underlying structure of the Customer, on average 80% of the code that uses the Customer object would not know about its constituents and would not need to be changed.

Of course, this is somewhat of a contrived example, but I hope it shows how concise, simple, and yet powerful Ruby's forwardables can be, but power comes at a cost: potential and ease of misuse! Enjoy.

Messaging Services

Written on 10:47:00 PM by S. Potter

Since my previous post was on the subject of comparing J2EE stack with lighter weight frameworks, I thought I would make some more points before Rails, et al. gets a beating for not having as much functionality as J2EE. I am sure my full-on Java friends will feel triumphant after my last posting because such a comparison would be null and void due to comparing apples and pears. Many of my Java friends haven't bothered to really research Rails, TurboGears or Django and not even Ruby or Python. I get the impression that they refuse to because then they might be pursuaded and be proven wrong, or at least not completely right!:) From an abstraction level the Java-heads might be right. J2EE can be a million different things to a thousand different people (yes, you read that right). Rails, TG and Django do not try to do this. They are fully loaded web application frameworks. Does this mean then that the conversation stops? I think not and I will tell you why. The vast majority of J2EE apps are written for web browser clients, Swing/JFC desktop apps or simply (mostly Java) headless services. Now lets consider, say, Rails. Maybe 98% of its clients are browser clients (I assume a much higher percentage than J2EE apps), but other clients are SOAP, XML-RPC or REST based clients which could be headless services or desktop applications. They could even be other web applications. So what does this mean? That comparing J2EE and, for example, Rails isn't quite as proposterous as Java-heads might first think. However, the one major area where there is an absence of functionality in, certainly, the Ruby world, in my experience (unless another Ruby-ite can point us in the right direction) is a Ruby based messaging service, like the JMS implementations out there. Some people say though that this is not a big deal, because you can generate Ruby and Python bindings for messaging services like Tibco, MQ and even JMS implementations. My problem with this is that there does not seem to be a good open source messaging service that isn't too Java-centric. In this article I give a measely 5 points to the Java-philes, but overall you are still far behind;) PS I am not suggesting that Rails should provide any type of messaging service functionality, only Ruby, Python or a non-Java centric enterprise-ready open source alternative to JMS, MQ or Tibco. If I were suggesting the former, then I would be completely contradicting my previous blog entry and I do not wish to do so. Thanks for reading.

J2EE vs. FastCGI

Written on 10:29:00 PM by S. Potter

After a month hiatus I return... Here is a link to a reasonable (meaning non-emotional) discussion of J2EE vs. FastCGI on the issue of scaling. One side note to the discussion that is mentioned several times in this blog entry is how FastCGI's API is extremely simple whereas J2EE's is extremely, well, not. It isn't a completely fair comparison since many developers use Rails, TurboGears or Django as a framework around the FastCGI base. However, in my experience with J2EE (even when it first came out), there was always an aura of complexity in the specification due to its inherent level of abstraction. Rails and TurboGears (sorry I have never used Django on any production-worthy project) both have an aura of simplicity from their inception. Another point I would like to make regarding Rails, TurboGears and Django is how different each of these frameworks differ in their approach to offer simplicity. While I am hopeful for the future of all three of these projects only time will tell if they will retain the simplicity they so vehemently publicize.

Search Engine Optimization: Devil or Saint? Science or Art?

Written on 10:35:00 PM by S. Potter

To many website operators, especially websites operated by individuals, small to medium-size businesses and non-profit organizations, Search Engine Optimization (SEO) techniques are black magic that have no rules. Others see SEO techniques as a sure-fire cure-all. While Google clouds its search algorithms in enigma to outsiders, there are cornerstones to implementing effective SEO campaigns. In this posting I wish to demystify the core building blocks websites should be basing their SEO strategies on. Recently while working on software and web development projects remotely for two clients I have also had the opportunity to work on four SEO projects of various scale since the beginning of this year. I have been aware of many SEO techniques (some of them I would term tricks), but prior to January I had only put them into practice on small personal web endeavors. In my experiences thus far I have found some fairly simple, yet very effective foundations to build SEO efforts on top of, namely:

  • Provide useful, well written content on your site. It does not need to be mountains of content, but it does need to be somewhat sensible and useful content. One way that you can promote content generation (depending on the type of website you have) without crafting it all yourself is to open up forums for users/customers/clients to discuss their experiences. However, in some cases, this could be a nightmare for smaller organizations or individuals that need to create a family friendly environment (in terms of moderating spam or inappropriate comments), but has been used to great effect on many well known websites including, of course, amazon.com. I should also note that technology (like CAPTCHA image validation) can minimize spamming.
  • Clean HTML documents that you would like indexed by search engines to contain the minimal amount of markup, styling and javascript code as possible. This is second only to the above cornerstone. As a software engineer with, most recently, a web development focus, I find too many websites that clients give me to optimize, need client and server code to be refactored in in a serious way. The idea is to reduce non-content code to the appropriate size that can improve keyword and search phrase ratios that most search engines most likely use to rank pages. I should note that we do not know exactly what ratios are used by search engines, but I have seen statistical evidence to strongly support the fact that certain ratios (or formulas that are proportional to these ratios) yield higher positions in web page rankings. If your web developer spends too much time learning the latest Javascript tricks or CSS hacks and not enough time writing more consice (but readable), more organized code, you should question whether they should be kept on for production work. Prototyping websites might be a better skill fit for them.
  • Know your audience [and keywords]. Three out of the four SEO projects I have recently worked on this year had highly flawed initial keyword/phrase targets, set forth by our clients. I was asked to promote these three websites on keywords very few people actually use to search for these business services. There are some commercial, free and internally developed tools and scripts that can help in analyzing the suitability of keywords. Making a list of not only the A-list keywords/phrases (the ideal candidates) for your website, but also popular B-list keywords/phrases, common typos and misspellings is essential to implementing an effective SEO campaign. Not only should you know your audience and relevant keywords and phrases to promote your website on, but you should be realistic about potential ranks for extremely popular keywords that larger players will, most likely, dominate for the first 3-5 pages of results. In these cases you should consider your options and evaluate if going after a 54th spot for a major buzz/keyword is worth the time and effort to get your site to that position when you could be getting first page hits with other more specific keywords and phrases.
  • Support search engine-friendly URLs on your website. If your website is dynamic in nature, choose web server technology that supports easily readable URLs. For example, most search engines will not index a web resource at a URL like the following: http://your.domain.name/dir?id=6434&type=restaurants&subtype=middle+eastern&zip=60640&order=ASC A website that has more than 1 dynamic HTTP parameter set in its URL is taking a risk that most search engines will not index it as search engines may assume it contains highly perishable information in the page. I have been able to translate unfriendly URLs to friendly URLs without asking the original web developers to touch the server code, although not all website URLs can be translated using URL rewriting engines such as Apache's mod_rewrite! Another gotcha is when web developers use the POST HTTP method when submitting HTTP parameter data using the GET HTTP method is far more appropriate. Since I am a software engineer with web development and SEO experience now I can advise clients on how to approach this and discuss best practice approaches. There are also other issues related to supporting search engine friendly URLs that would probably require a whole book let alone a long blog posting to fully treat the subject.
  • Do not blatantly violate the spirit of the search engines' code of ethics. There are some techniques that are various shades of grey, but you should stay clear of all techniques that are darker than 50% grey. If the search engines do not notice, your competitors might and they will be sure to notify search engines if you are stealing their thunder using questionable techniques. In theory, Google does not believe in blacklisting individual sites and prefer to optimize their algorithms such that the website drops in rank significantly or is no longer listed. However, Google will blacklist when websites are blatantly violating the spirit of their rules. Most recently Google has removed BMW's German website from search results on certain key words as they overstepped the mark in SEO creativity: http://www.alwayson-network.com/comments.php?id=14809_0_5_0_C
As you have probably figured out from my ramblings above, there is no clear winner. SEO techniques are neither black magic riddled with negative consequences nor sure-fire cure-all solutions. All I suggest is treading carefully using tried and tested techniques with room for some experimentation of your own to determine your own conclusions if you are going to undertake this task yourself. Alternatively, if you hire an SEO consultant make sure they are neither extreme.

IlliPy - C-U's Python User Group

Written on 11:06:00 AM by S. Potter

For those of you that don't know, I live in Champaign-Urbana (yes, birthplace to both Jennie Garth [90210] and Roger Ebert [film critic]) and have yet to meet enough techies. So my mission currently is to start a Python User Group for C-U. I have named it IlliPy, for reasons that should be blatantly obvious to the avid Python freak living in C-U. While I would like the group to have a monthly or quarterly (depending on number of people interested) tech talk/presentation and perhaps a book club, I also wanted to initiate some open source projects with other group memebers. These open source projects do not need to be purely Python only, but need to be related. Another angle I wanted to pursue is to also include a couple of talks on Ruby vs. Python, because there is a lot of overlap plus a lot of significant differences too. In fact, Python incorporated a Ruby feature into 2.4 because it worked so nicely in Ruby. If you are not a Python freak, but would like to get involved, fear not! Please come along to find more out about Python. I should also state for the record that you do not need to be a programmer by profession to participate. There seems to be a number of non-professional Python programmers out there (see this podcast for a Python fan that is not a programmer). Our first kickoff meeting is scheduled for Wednesday April 26, 2006 @ 7pm at Giuliani's (where the old Green Street Coffee Shop used to be and next to Murphy's Pub). To be informed of updates to the kickoff meeting or other events, please feel free to subscribe to the announcement-only mailing list.

Iron Python

Written on 4:10:00 PM by S. Potter

I just wanted to post the link to IronPython's download page since I keep mislaying the link and I revert to visiting the old ironpython.com website, which is totally unmaintained presently. Iron Python Download Site

Tax Day, Psychology & Pragmatic Tools

Written on 12:25:00 PM by S. Potter

Since I moved to the US I have always been struck by how wonderful the weather is on Tax Day. Since 2000 I have procrastinated doing my taxes until the VERY last minute (or at least last two days). I have traveled from coast to coast and lived in the midwest all since 2000 and every place I have been on Tax Day is a beautiful sunny, not too hot, not too windy, simply beautiful day! This year the regular Tax Day (April 15) landed on a Saturday so the IRS generously gave us until the following business day to e-file or mail our returns. Technically I finished filling in my tax forms on Sunday (April 16, 2006), but I only mailed the returns and added the extra $1,000 to my Roth IRA for 2005 this morning. Did you know that the limit is $4,000 this 2005 and 2006? Hurry you still have time to make the contribution. Perhaps the reason that the weather is so beautiful on Tax Day is that we appreciate it more knowing we have another 365 days until we need to do it all over again. Is it all psychological? I have been wondering this with respect to my work recently too. Over the last 2 years I have been increasingly more a Python head and less a Java head. Now I have started a medium scale website in Ruby on Rails and am having a great time doing so. The reason I have slowly been withdrawing from Java towards Python and Ruby is that I have found myself actually enjoying my work again on a low-level. It is honestly a pleasure to write Python and Ruby. Don't get me wrong I still need to fix some Java code in older projects that I still look after, and I am not saying Java is a bad language. In fact, compared to C++ (one of the languages I used on my first three jobs) it provides many productivity enhancements to writing servers in J2EE and even GUIs in Swing/JFC (as compared with MFC not comparing .NET here). C++'s Boost libraries are great if you are still a C++ head, but I think they arrived way too late in the game. For me Python and Ruby provide me with tools that psychologically enhance my state of mind while writing code. For example, I do not need to wait for byte compilation the same way I need to wait for a Java project of a similar size to compile. This means I don't loose my train of thought between writing the code and running the tests while checking my emails or getting a refreshment in the meantime. It also means that it simply takes less time to complete. The more time I have for writing and running tests the better in my opinion. I felt in Java and C++ that waiting for the compilation process (especially C++;) made me loose some of my gusto and determination for writing more tests. By providing pragmatic tools that enable me to write more tests without negative consequences, Python and Ruby promote more test writing and result in me having a large collection of test cases to run on a continuous or regular interval. One of the other major benefits of using Python and/or Ruby is their advanced interactive shells (ipython and irb respectively) that neither Java nor C++ possess in a serious fashion (e.g. Groovy, Jython, JRuby, etc. are not truly interactive Java shells in the same way). These two interactive language shells are huge time savers and promote true prototyping in the shell. Prototyping in this fashion makes sure that prototype code is truly thrown away except for the lines of code you have made the extra effort to cut n paste (i.e. the lines that worked the way you needed) and thought about which to keep. Again by using these pragmatic tools we are psychologically driven to doing the *right* thing for the project and learning the APIs more quickly. The third major benefit, and in my opinion the most overlooked (albeit quite new) attribute of these two pragmatic language environments (Python and Ruby), is package management with version/dependency management and discovery mechanisms built-in. In Java you have Maven, which attempts to do this for you in a round about way (and do not get me wrong Maven is very good for the Java world), but this is not a standard, either defacto or otherwise. Sure Python's Eggs (and the associated setuptools module) aren't true Python standards yet, but they soon will be and are very widely used both in open source and corporate worlds. Both Ruby's Gems and Python's Eggs provide developers, development teams and system administrators as a whole an easy way to manage multiple versions of code packages and components without worrying about setting environment variables and/or server container arguments appropriately, which might be out of the developer's control. The less time developers need to spend time on configuring their environment the better. Eggs and Gems streamline much of the configuration processes significantly as compared to Java's JARs limited functionality. While my personal experiences catalogued above isn't necessarily applicable to all developers, overall I think one of the keys to boosting development productivity in a language is understanding on a psychological level what developers NEED to work smarter!

Kaizen

Written on 12:13:00 PM by S. Potter

Kai-zen. What a fantastic word! A recent addition (ok, I am behind the times - more like 2 years ago) to English dictionaries is 'kaizen'. For years the Japanese have been using this wonderful word, which simply put means 'continuous and incremental improvement' or 'change for the better'. How wonderful to have one word to describe something that can be so layered in meaning? If you read the wikipedia entry you will find kaizen is about attaining the following goals:

  • eliminating waste*
  • just-in-time delivery
  • standardizing work
  • using only appropriate equipment
  • leveling load
  • plus others...
(* waste is defined as "activities that add cost but do not add value") In fact, kaizen is about taking a process, system, product or service apart and putting it back together again with the sole purpose of improving it. Kaizen is a whole philosophy that does not only encompass mechanical or purely business principles, but also humanizes processes and promotes respect for people. Without this basic respect honored there will be no continuous improvement at all. Taken directly from the wikipedia...
Importantly, kaizen must operate with three principles in place: process and results (not results-only); systemic thinking (i.e. big picture, not solely the narrow view); and non judgmental, non-blaming (because blaming is wasteful).
Hopefully now, you will understand why I fell in love with this word almost immediately. I am surprised that developers haven't started using this word, such as 'kaizen programming' or 'kaizen development'. I do think that Agile development methods and practices are moving very close to the essence of the meaning of kaizen. Perhaps now we should take these processes apart and reassemble with the purpose of improving them, ad infinitum. Unfortunately due to human error and nature we will never find the perfect methodology, but we can use common principles we all believe in to move us closer to our utopia, whatever our poison is (in my case software development). Taking a fresh look at CMM and Six Sigma, I wonder if this is where both processes originated from. If so, over the decades both have devolved and moved away from their founding principles. I am not attacking the CMM approach or Six Sigma methology at its roots, but pointing out that in industry today, their implementations (in my experience) have lacked any notion of humanizing the process. I have found the contrary true, they both try to dehumanize processes, which contradicts kaizen fundamentally. Today I decided to marry my former favorite word, mojo, with kaizen and registered kaizenmojo.com. Not completely sure what will be on this site, but I do have some software development productivity tools that need a home, so this might just be the perfect place. So continue improving...

Agility: Pair Programming & Quality

Written on 12:40:00 PM by S. Potter

I have been working on Agile development teams on and off over the last few years. Not all the "agile" teams I have worked on have been as disciplined as they really ought to have been, but I was wondering what agile developers thoughts are on effective pair programming practices? I know some of my ex-colleagues are quite happy with having one programmer glance at the other programmers code (after the other programmer designs and codes it by themselves) and write tests to see how to break the code, but I am not sure this is good enough for the sake of quality. I personally enjoy working with developers at all levels of experience assuming they are positive, friendly people or at least professional. Now I also know a number of developers that don't fit the "positive", "friendly" or "professional" molds, so I was wondering how to handle those situations effectively without compromising the software's quality early in the process. At the end of the day I am focussed on delivering high-quality software to clients because my job is on the line if I screw up and also my bonus depends on it, even if I don't screw up. The company I presently work at, Finsignia LLC, is more disciplined about agile principles than other organizations I have worked at previously. In fact, Finsignia is the only company I have worked for that specifies, in their contracts with clients, quantitative measures or ranges to define the level of software quality we will deliver to the client. For example, > 98% unit test coverage, SMI ranges, etc. Quantitative measures are even more important for the project/developer leads (which I am one). While we have a lot of automated tools that we incorporate into our continuous integration processes to measure our quality quantitatively, I don't like that thought of having to refactor something later when it should have been caught earlier on in the process. Presently the developers on my team have a blast, but I know at times people just can't get along in a positive way for various reasons. So I would appreciate any wisdom from people who have had to deal with this type of situation. I invite you to leave me your pearls in comments to this blog.

Trac and Alternatives

Written on 11:41:00 AM by S. Potter

I have been using Trac to help organize my development projects for about 6 months now and quite happy with it. I haven't yet found an alternative that can help small teams work more efficiently on agile projects.

So I would be interested to know if anyone out there presently uses something like Trac, which could be open source, free or commercial or even in-house proprietary software to help organize agile software projects? What is it you like about the products you use and why?

I would summary my experience with Trac as follows:

Pros

  • Easy to use
  • Simplistic and organic organization
  • Great to have integration of changeset views and comments, tickets and even wiki pages

Cons

  • Missing user story aspect of project audit
  • Lack of linking between milestones, components or versions

BaseCamp

Written on 10:53:00 PM by S. Potter

Basecamp project management and collaboration I just signed up for a free BaseCamp account. This is the web application that generated a lot of buzz last year because it was developed by Ruby on Rails (RoR) creator David Hansson, while he was developing RoR. BaseCamp is supposed to be a web-based project management tool. However, after watching David's talk at Roskilde (on rubyonrails.org) I guess "project management" is too much of a supercharged phrase to be applied to this web application. David also seemed to imply that it is more of an application to aid collaboration rather than manage it. I thought it would be completely amazing considering all the hype. Currently I am only luke warm on the product, but perhaps I need to use it more before I really start to like it. Up to now this is what I have noticed.

Pros

  • Nice design
  • Nice AJAX usage (mostly)

Cons

  • Waay too simplistic
  • Not enought functionality