Cloud Computing, Part 2: Law, Privacy & Maturity

Written on 9:59:00 PM by S. Potter

At the CEO roundtable event I mentioned in my previous blog post about in Cloud Computing, Part 1 the other major business consideration that came up was how to protect your business' (and/or customers') assets properly. Ultimately anyone that wants to build a solid foundation for their business needs to tackle this well to be successful long-term. One person asked the speaker to elaborate about a recent case where a UK bank used servers in datacenters inside India where a datacenter employee stole customer information and used it to effectively steal money from these people's accounts for his own benefit. This could happen in any country, however, in this case this datacenter employee actually didn't break any laws inside of India (or broke one very minor law that had a very limited penalty). Ultimately this lack of relevant law and regulation in a foreign country is the major risk for a business using offshore datacenter resources. Unfortunately I didn't hear the name of the bank involved to be able to find a news article about this incident, however, the speaker had heard about this incident (it wasn't just a figment of the questioners imagination) and his response (I am paraphrasing here) was:

Countries with the most mature infrastructure, laws, regulations and standards regarding data and information law (as well as privacy) will probably be the places that larger firms will invest in for sensitive and mission critical systems of their organization despite the likely higher cost of building.
As you determine whether you are going to be a cloud consumer or provider in the future, make sure to keep an eye on these risk factors as some executives seem to only focus on the raw bottom line without conducting a thorough risk analysis to see if the cost savings are really worth the higher risks. Choose the best cloud computing partners for all your cost trimming AND risk mitigation needs.

Cloud Computing, Part 1: Delivery Models

Written on 9:10:00 PM by S. Potter

Today I went to a regional CEO Roundtable lunch talk given by Professor Roy Campbell at UIUC on Cloud Computing and why executives of any kind of business should care.

While I am not new to the concepts of cloud computing and multi-tenancy SaaS, since I have consumed various cloud services and designed multi-tenant architectures for two previous clients. However, there was one particular "overview" slide that Professor Campbell showed that got me thinking a little. It was titled Delivery Models.

It was a text slide with bullets, but I think a diagram (or two) could have communicated to the higher-level audience more than most of the first half of his presentation could have (see below and let me know if it is more effective).

I especially liked Professor Campbell's categorization of the Delivery Models. Until this lunch time I had really only considered there to be two cloud delivery models: SaaS and PaaS. I wasn't making a distinction between Amazon services like S3 and EC2 and that of Google's AppEngine. On reflection, I am glad I took two hours out of my busy Friday if only to consider the distinction made in this talk.

In a nutshell the slide has three major bullets:

  • Software as a Service
  • Platform as a Service
  • Infrastructure as a Service

Nice! Google AppEngine, essentially being an application container (that supports various frameworks now), is a platform, whereas S3 and EC2 provide infrastructure on demand.

Now unfortunately this is where Campbell left it. For those of us not unfamiliar with cloud computing, we can see the importance of this distinction so that we can develop a more solid business plan (for those of us that might be interested in using the cloud to generate revenue from). It may also help technically savvy people that aren't familiar with the cloud.

Hope this helps people make a choice between container services like AppEngine or Heroku (platform) and those such as S3 and EC2 (infrastructure) from a consumer side and perhaps those devising business models to focus on their core competencies.

Next time I want to talk about privacy, legal and compliance issues surrounding Cloud Computing as well as how business should weigh the risk of outsourcing their data handling (however that happens) to countries will less mature laws, regulation and oversight. Perhaps cloud governance would be the most appropriate term?

Alan Turing Can Now Rest In Peace

Written on 9:42:00 PM by S. Potter

While I am by no means the biggest fan of the New Labour government in my homeland (mostly due to Blair's sociopathic reign), I have to give Gordon Brown props (or at least his aids) for giving credit to Alan Turing and saying Britain, as a country, fucked up big time (sorry, I really couldn't think of another phrase that matched the intensity - I'll replace it if you suggest a good alternative).

You can read the Downing Street blog post here:
Treatment of Alan Turing was “appalling” - PM

Thanks from this particular graduate from University of the Manchester's Mathematics department who now works as a software engineer who also happens to be gay!

"Ta luv", as the Manc locals would say.

Faker, Ruby on Rails Snippet & Generator

Written on 7:07:00 PM by S. Potter

Hey blog readers (if there are any left), I know it has been a while, and I know I have left you high and dry, so today I decided I will share some code and software development thoughts with you. I am moving to a more snippet oriented blog posting style from now on. Previously I preferred long ramblings about the political ideology of programming languages and even how Ann Coulter was _more_ comprehensible when I used my Metafusion Crypto library on her words (actually that was a marketing lie - she sounded just as incomprehensible using Metafusion Crypto as she did when not using it). Of course, this was all in 2006-2007 when blogging was all about spewing small tidbits that taught the reader nothing, so I had to be different and attempt to find a reason for things. Now things are swinging so I am trying to do the opposite from the norm again. Ok, on with it. Over the last week I have been using the Rails 2.3.4 "seed" mechanism, which in essence attempts to provide 80% of seed-fu's features inside Rails core. In addition on my personal projects I have been using machinist as my factory framework of choice (although I am forced to use factory_girl for my clients), through which I was introduced to faker. A beautiful gem indeed. So to cut a long story short, I have found myself doing the following a hell of a lot in my brand new Rails 2.3.4 projects this week: # in db/seeds.rb def load_seed_files! seed_files = Dir["#{RAILS_ROOT}/db/seeds/all/*.rb"] + Dir["#{RAILS_ROOT}/db/seeds/#{RAILS_ENV}/*.rb"] seed_files.each do |file| puts "Loading seed file: #{file}" load(file) end end load_seed_files! # When I add files to the db/seeds/all directory it seed ALL environments # When I add files to the db/seeds/{development,staging,test,production} directories it seeds # for only that environment. Note: I wouldn't recommend using seeds in just staging, test or production generally. If you need seed data in these environments you _probably_ need them in all environments OR (in the case of the test environment) you need to use a factory framework like machinist or factory_girl. Now where is faker? Well it's what I use inside these .rb files where faker comes into play: # Faking a company name >> Faker::Company.name => "King-Ruecker" >> Faker::Company.name => "Cartwright, Prosacco and Smith" >> Faker::Company.name => "Pagac, Adams and Barrows" >> Faker::Company.name => "Satterfield and Sons" >> Faker::Company.name => "Blick-Deckow" # Faking a person's name >> Faker::Name.first_name => "Ruthe" >> Faker::Name.first_name => "Jameson" >> Faker::Name.first_name => "Avery" >> Faker::Name.first_name => "Jordan" >> Faker::Name.first_name => "Ruthe" >> Faker::Name.first_name => "Winona" >> Faker::Name.last_name => "Brown" >> Faker::Name.last_name => "Hayes" >> Faker::Name.last_name => "Jerde" >> Faker::Name.last_name => "Lindgren" >> Faker::Name.last_name => "Considine" >> Faker::Name.last_name => "Rodriguez" >> Faker::Name.last_name => "Turner" >> Faker::Name.name => "Mekhi Lebsack" >> Faker::Name.name => "Esteban Brown" >> Faker::Name.name => "Velma Crona" >> Faker::Name.name => "Rollin Zemlak" >> Faker::Name.name => "Casper Sanford" # My favorite is coming up with BS (yes, BS is what you think it is): >> Faker::Company.bs => "redefine ubiquitous e-commerce" >> Faker::Company.bs => "synergize viral e-business" >> Faker::Company.bs => "innovate real-time experiences" >> Faker::Company.bs => "aggregate scalable systems" >> Faker::Company.bs => "e-enable back-end functionalities" >> Faker::Company.bs => "implement 24/365 models" So the point of faker, if you haven't already figured out is that it generates data the way you would probably have in production so you are generating meaningful data in your development environment(s) where you want to show your client what the interface will look like with data inside. Now a word of caution: I would not advise you to use this in an end-user facing test environment. I sometimes call that staging or qa, but ultimately it is an environment where real end-users or clients test out the features in a stable environment (not development or continuous integration environment) so they can sign off on your stories. In these environments you should let the user create the data themselves using the admin forms or whatever UI you provide them because you want to see how they will break the system (and we all know they will somehow). It gives you a much better indication of how things will end up working in production, which is very important. Thanks for reading and let me know if this is yet again too rambly? How does it happen? I have the best intentions to just blurt out tidbits with no nuance, yet I always end up rambling on about things and give it a thought process (well sort of). Anyone want to be my editor to strip out all thought and insight so I can be just blog brainless "tip" blog posts like everyone else in the Rails community that wanted to be a micro celebrity (trust me I couldn't handle that kind of paparazzi attention and I don't even have a Macbook Pro to sling over my shoulder in a fashionable messenger bag)? I can't pay you anything, but you could share in my measely $0.63 that Google Adsense pays me each month (actually they don't actually pay until a threshold is met). You could buy one third of an MP3 from iTunes each month. Any takers? What an awesome deal! No really! PS Coming soon a simple generator and Rails application template (or applate as I like to call them) that will set you up with this so you don't have to copy and paste all the time. It'll be on GitHub.com/mbbx6spp. Ta ta. Update: As Damien in the comments pointed out indirectly, I really didn't do Faker justice. Please read the Faker RDocs for more information.

Design Patterns Book Club

Written on 10:05:00 AM by S. Potter

I just discovered that Hashrocket has a video book club where people can join in the conversation with a live videocast on USTREAM or watch older episodes on their own schedule. They are currently going through Russ Olson's Design Patterns in Ruby. This is a great idea, kudos to Hashrocket. I watched the latest episode on the Strategy pattern and had a couple of thoughts I would love Desi and the rest of Hashrocket to consider for subsequent episodes:

  • I liked that this specific episode wasn't very long (I know you were a little worried)
  • I think discussing anti-patterns that could be solved with this pattern you are discussing the episode is very valid (as the person sitting next to Desi said). Perhaps that could be explored more. Note: I have only seen that one episode so perhaps you do this in other episodes much more, but if you don't please consider it, because I feel that anti-patterns are very important to identify. Think of the design patterns as the treatment or medicine and the anti-patterns as the symptoms. Physicians need to be acutely aware of the symptoms and symptom combinations to be effective at prescribing the best course of treatment or medication. Therefore we should be teaching software engineers in the same way.
  • Also please consider putting the name of the design pattern under discussion in the title of the video for archival purposes.
Thanks goes to Hashrocket for the great idea and work of putting it together! It's about time the Ruby community starts to mature w.r.t. language-neutral concepts and these efforts will hopefully help those that came into software development from a less formal software engineering education path (I used to be one of them and had to learn from scratch myself) with a little more assistance. It was pretty lonely bookwork and experimental projects for me to learn this stuff when I first started out.

Google Wave is not the answer, XMPP is!

Written on 1:00:00 PM by S. Potter

Since yesterday afternoon (my time) all over the Twittersphere (and blogosphere) I kept hearing a ton of great reviews of the Google Wave API/protocol and how it will revolutionize communication and collaboration on the web. This morning I watched the Google Wave demonstration from yesterday and I have to say I am completely underwhelmed considering everyone else's reactions were ridiculously positive and had nothing measured or thoughtful to say about it. You can find my initial reaction of it, "Google Wave is just a simple XMPP extnesion", on my tumblelog. As someone that has had to write XMPP extensions for clients and my own internal apps, servers, API interfaces (mostly distributed monitoring of services, but a few more for media processing) I am saddened that anyone would think the current Google Wave protocol (which is just an XMPP extension) is extraordinarily ground breaking. There have been a number of existing extensions that tried to do VERY VERY similar things already and specific features of Google Wave were already exactly defined. What made the demonstration of Google Wave yesterday appear so impressive were the applications: the wave client and the web embedding. The responsiveness of the web application really made it stand out, NOT the underlying Google Wave protocol/API. For those who have never really gotten their hands dirty with XMPP in any real technical way, I expected they would not really know what it already provided in XMPP. Many "geeks" hyping up Google Wave seemed to think XMPP only supported instant messaging. That just happened to be the most popular and apparent XMPP extension, which many people call Jabber. XMPP itself is a lot more far reaching than just an instant messaging protocol and the fact that Google Wave was based on top of XMPP should tell you how flexible it is. Yes that is right, Google may not have really made a big deal about it (why would they if they can get extra credit when it isn't due), but Google Wave is just an XMPP extension and compared to other extensions I have had to interface with, not a particularly complicated one. Sure the applications that Google demonstrated as part of the Google Wave presentation were nifty for personal communication, but are they really ground breaking, multipurpose, or ubiquitous? No, it is only as ground breaking, multipurpose and ubiquitous as it's applications make it. Let us all now return to earth for a moment and praise Google for the things they do really well like web applications, but let us not pretend that Google is the mover and shaker in terms of the visionary of future underlying protocols. They still advocate developing HTTP applications for the future. HyperText Transfer Protocol (HTTP) is virtually on every digital device made by man today, but does it really lend itself to providing the distributed service oriented architecture we should be striving for and the protocol of the future? REST is not about the HTTP protocol. RESTful APIs can be creating in a number of ways and through a lot of different protocols. My take is (from a technical level) XMPP ought to be the protocol of the future, NOT XMPP on just the back-end and then adding tons of bloatware in between the client and the service just so that we can create an API or UI on top of an antiquated protocol (HTTP) that has seen better days. Anyone want to create a truly modular XMPP "browser" for the new "web" with me? Email me at susanpotter dot net if you want to join the future. UPDATE: Please subscribe to the "Join the Future" Google Group if you want to get involved in the XMPP browser project.

Related Links

Erlang doesn't support currying? News to me!

Written on 4:13:00 PM by S. Potter

I keep seeing developers who only glance at Erlang and think that currying isn't supported. Nothing could be further from the truth...unless you are defining currying support only in terms of specific syntactic sugar in a language for currying. So let us step back and define what currying support really means:

Currying is the process of transforming a function that takes multiple arguments into a function that takes just a single argument and returns another function if any arguments are still needed. http://www.haskell.org/haskellwiki/Currying
Let us look at a simple (but hopefully not too contrived, albeit it _slightly_ contrived) example:

%% currying_example.erl
-module(currying_example).

-compile([export_all]).

multiply(X, Y) -> X*Y.
doubler() -> fun(X) -> multiply(2, X) end.
In your erl shell try the following:

1> c(currying_example).
{ok,currying_example}
2> D = currying_example:doubler().
#Fun
3> D(6).
12
4> q().
ok
As you can see in the module code above, Erlang supports what is called anonymous functions (e.g. fun (X) -> somePredefineFunction(OtherVarOrConst, X) end), which then allows us to manufacture single argument functions in a way that is clear and expressive in the code IMO. For those that haven't been following the functional programming debates over the last week or two around the blogosphere, please see the following blog posts (some are pretty funny and will make you smile at least twice with their intended absurdity): Previous posts on this blog that are related are: I will try to put together a few snippets on currying support in Python, Javascript and Ruby tonight after my Greek class (which I am late for now) if I have time as a quick exercise:)

Twitter4R 0.3.1 and the future

Written on 12:16:00 PM by S. Potter

Hi Twitter4R users, I know I have been rather absent in terms of my Twitter4R contributions and ongoing development, but despite my absence I have _finally_ officially released 0.3.1 with a lot more new features and a few bug fixes more than I initially expected to release 0.3.1 with. Some of the goodies included in the official 0.3.1 release are:

  • Added raw Twitter Search API support. Meaning you can use any of the HTTP parameters the Twitter HTTP-based Search API will expect. E.g. twitter.search(:to => "twitter4r", :rpp => 10) or twitter.search(:q => "twitter4r", :rpp => 15, :geocode => "40.112186,-88.207592,250km")
  • Profile, theme API coverage on Twitter REST API.
  • Social Graphing API. E.g. twitter.graph(:friends, user)
  • Added "reply" support at all levels (low-level to model-level)
  • Bugfix: the infamous URI.encode to CGI.escape change.
  • Rate limit information (still needs to be able to support the HTTP header method, which I actually suggested to Twitter developers some time back, but they only recently added it when my bandwidth for OSS contribution was minimal) :(
  • much more
Now the future of the Twitter4R project. I have made a LOT of mistakes with this Twitter4R OSS project venture, so I am going to try to fix a bunch of these things:
  • Create a new set of Git repos on GitHub that separates the education portions (screencast, etc.) from the core source code of the project so that people do not need to download ridiculous amounts of data to their local drive just to fork Twitter4R on GitHub as it required at present. Announcements about this will be coming soon!
  • Have a consistent issue tracking system for Twitter4R. Previously it has appeared inconsistent to users of the Twitter4R library where to submit defect reports or new feature requests to. So I have had to field these requests from the Google Groups mailing list to personal emails and also @twitter4r replies on Twitter. I plan on using GitHub's new issue tracking system for this as it seems a logical extension.
  • I plan on documenting things a little more obviously on the official website from now one. It appears many users did not want to go to the RDoc unless specifically pointed to it, even though a large majority of the library is fairly well documented in the RDocs.
Please be patient while I fix these process/marketing/workflow issues of the project and in the mean time I will also be tackling OAuth support as well. Again thanks for your patience thus far and tweet me on twitter at either @SusanPotter or @twitter4r if you have suggestions or questions about the direction. If you have specific queries on using the actual API, please join the twitter4r-users Google Groups mailing list and search the archives for a solution first and then if you can't find a solution, please post a message giving the group members your environment details and the version of Twitter4R you are using. Thanks! Susan Twitter4R Developer

The Scala vs Erlang Debate, Part 2: The Geek Off

Written on 3:47:00 PM by S. Potter

In the first part of The Scala vs Erlang Debate article series, I discussed various ways to go about adopting either Scala or Erlang and in what scenarios it made sense to go with one over the other from the manager's perspective. In summary the major advantage of Scala is that it runs on the JVM so it can access pre-existing Java libraries your enterprise has already developed. This is also Scala's major disadvantage because it makes a rather large assumption (which in my opinion would be incorrect the majority of the time) that the Java libraries being used are thread-safe. Erlang on the other hand doesn't allow you to integrate in the same OS process with Java libraries, because Erlang was built from the ground up with scalability and fault-tolerence in mind and ideally you shouldn't put Java into the run-time stack because it compromises Erlang's fault-tolerance features significantly. Instead the preferred approach when writing the biggest bottlenecks of your system in Erlang and integrating with pre-existing Java libraries/system is to use SOA to create separate, but integrated systems that are easy to swap out and upgrade at a later point. In my view, you should be refactoring your enterprise architecture by defining services anyway, since this will decouple systems and remove a great deal of headaches in IT maintenance and deployment. Today I wanted to geek out and differentiate the two environments on a more technical level for the middle/technical managers.

Scala: Technical Overview

Scala is an objected oriented, statically typed language with many functional programming capabilities built-in. However, it must be noted (because few Scala documents do this) that Scala is hybrid actor programming language and therefore a number of assumptions cannot be made. So by being a hybrid functional language it losses some of the benefits. As well as it's Java run-time support on the JVM it is also supported in Microsoft's Common Language Runtime (CLR) so .NET integration, in theory, is a breeze. Again this also assumes that the .NET libraries you use from Scala are thread-safe. On a related topic, .NET also has F# available, which is Microsoft's attempt at a hybrid functional programming language also. I have no comment on F# myself, since I have zero experience with it. Please chime in on the comments section if you have experience with F# and are able to contrast it with Scala.

Erlang: Technical Overview

Erlang on the other hand is a dynamic, purer functional programming language that supports light-weight processes in its own run-time environment (separate from OS processes), which send messages to each other asynchronously. One major side effect of being purer in its actor treatment is that there is virtually not state. There is no way to pass "state" around (a couple of minor exceptions), but there is a lambda calculus optimization that can be taken advantage of that will ease away our troubles called tail-recursion. Tail-recursion only partially works in Scala. For many developers that only have experience in procedural and Object-Oriented (OO) programming languages like Java, Python, Ruby, C++, C, Perl, C# this sounds completely insane. Two years ago I may have agreed with you, but today life is much simpler without state in my Erlang travels than when I had to write thread-safe Java code and I wish I had discovered this concept earlier. I will not pretend this notion didn't take some getting used to but today I see things a lot clearer and I am thankful for that clarity.

Scala: Concurrency

Concurrency in Scala is modeled using a concept of actors, except there are two types of actors: thread-based and event-based actors. Since Scala was implemented on top of the JVM, which is stack-based, Scala had to compromise on Erlang's one actor approach. This is because native threads, which the JVM uses for concurrency support, are much heavier-weight than Erlang's lightweight processes thus reducing the number of thread-based actors that could be launched inside one JVM. So Scala created this concept of a very peared down event-based actor, which does not have all the functionality of thread-based actors but does not incur the penalty of needing to be backed by a native thread. To be as complete as I can be in this section I should also note that Scala treats everything as an object, even though it has these two types of actors. Actors in Scala are just objects. In turn this may create duplication between objects and actors.

Erlang: Concurrency

The Erlang VM was built from the bottom up with lightweight processes (a.k.a. actors), therefore it is stackless and much better at lightweight concurrency. Erlang also has a built-in mechanism for load balancing these lightweight processes across all available CPU cores. This means Erlang has a massive advantage over JVM based functional programming languages at getting the most out of multi-core hardware, which is at the crux of the problem domain that software needs to address going forward.

Scala vs. Erlang: Performance

In terms of linear performance (sequential) Scala has been shown to beat Erlang in certain cases. However, concurrent performance consistently shows Erlang beating Scala, sometimes by wide margins on multi-core hardware. Not surprising considering what we learned in the concurrency sections above. When considering the problem we are trying to overcome - optimizing software on multi-core hardware - it seems clear which of these benchmarks are more important! :)

Conclusion

I think it is clear, which side of this debate I firmly stand, but I have tried to demonstrate both good and bad traits of each programming language from a technical perspective that might translate into a business case or reason for choosing one over the other in your organization. If I have time I might do a part 3, which will talk a little about syntax and DSL support. This might round out the argument since it could be argued that Scala wins in the third part by a hair, but all is not lost for Erlang if the underlying problem you need to solve is related to optimizing software for modern hardware and building robust systems at the same time. My vote still goes to Erlang despite the lack of built-in features that make designing DSLs easier, but it is not difficult to build your own DSL engine in Erlang either.

Limitations of this series:

There are quite a few topics I left out of this post that might be of interest for some applications or services. Namely:
  • Hot swapping: Erlang has great hot swapping capabilities for production environments that minimizes or possibly eliminates downtime of systems. This feature in Scala is not close in capability or reliability at this point.
  • Garbage collection: without getting into too many details Erlang probably wins this section by a hair (not significant to write home about). Long sweeps will not be a problem in Erlang and only a problem in JVM languages on rare occasions.
  • Distributed nodes: The clear winner here would be Erlang because built-in to the environment are all the multi-node features that make distributed systems a breeze. Scala on the other hand has different frameworks and libraries to facilitate distributing nodes of logic on different hosts.
  • [Added Nov-2009] Frameworks: As far as I know, Scala doesn't have an OTP killer, not even a contender to compete with Erlang's Open Telephony Platform (OTP). Sure it will take some time, even Haskell, which has been around much longer than Scala is slightly jealous of OTP (I saw the way Don Stewart looked at me when I asked him if Haskell had framework similar to OTP;) ). Supervision trees along make it stand out as the best thing since sliced bread, at least for this particular software engineer.

Resources:

Corrections (added February 2010):

  • Erlang is not a pure functional language. However, it does not have a polluted (impure) treatment of actors like Scala:) Some think this is a good-enough approach to be able to leverage the JVM, when running on the JVM is really important, perhaps 4 times out of 5 it might be "good enough". I just wonder if it really matters for most projects that you run on the JVM!?
  • "Scala's lack of tail recursion is entirely due to a trade-off. The JVM doesn't do tail recursion itself. It is possible to add tail recursion on top of the JVM using various tricks, but they are expensive. SISC does its own stack based heap and Kawa uses trampolining." -- James Iry

The Scala vs Erlang Debate, Part 1: The Managers Overview

Written on 9:37:00 AM by S. Potter

Note: The discussion is primarily between Scala and Erlang, but other languages could potentially be substituted in the arguments for and against. Scala stands for any language that attempts to solve the multi-core issue software developers face with threads and Erlang is a language that has light weight processes which solves the biggest issue of going with a multi-process solution in software. The debate started with the realization that Moore's Law died a while ago. Tomorrow (as we see it now) is destined for multi-cores. A problem of massive proportions now faces software engineers, since hardware engineers changed the rules on us. Now I am not giving hardware engineers a hard time, I am simply pointing out that there has been a massive shift in the way hardware is built, which directly affects the performance and reliability of the software which runs on it. It is not a transparent change that software engineers can ignore. With that background laid out, as software engineers, we must appreciate the problem we now have to tackle, both server-side and client-side developers alike will need to tackle these issues promptly. However, this article only considers server-side software as there is where my specialized expertise lies. This part of the article is the high-level. I'll get into the specifics in the second part. So the question now is how do we leverage the advantage given to us by new hardware progress of multi-cores? There are two fundamentally different problems to solve: software products/projects that already have large complex systems already built in languages that support threads half-decently such as in Java and JVM languages including Scala; but there are also those startups that don't have baggage and can start with a better long-term solution by using a true concurrent language like Erlang.

The Legacy Enterprise

The problem with utilizing JVM threading (whether directly in Java or via other languages built on the JVM such as Scala) is that third party code you use or existing Java/JVM code written isn't necessarily thread-safe. You might have all your business entities and models defined in Java, but can it really be threaded? The pain of migrating or porting your existing system from a JVM language to another truly concurrent language like Erlang might be much greater than dealing with threading issues in just a few key bottleneck areas that will increase throughput and CPU utilization enough. In this case I doubt many pragmatic engineers or architects will advocate starting from scratch in a language you know little (if anything) about. However, identifying those key pain points might take some time, but will be a necessary cost if you are serious about solving this problem in the near-term using a solution like Scala. However, what these engineers and architects should stress to management is that this is likely to become a monkey patch. There will be a tipping point where moving to a pure concurrent language like Erlang would be more beneficial that sticking to the monkey patch and living on the JVM forever, no matter what Sun tries to tell you. There are hybrid solutions for the medium-term, that depending on the project/enterprise specifics may be a good solution until you are able to fully port the code to a langauge like Erlang (or whatever the trend may be at the time you get to it). SOA principles can be used to design and define a meaningful architecture that will declutter the system and allow you to tackle just the bottlenecks of the system in an Erlang port/rewrite and still give you (the manager) a low risk PoC project so you can evaluate Erlang in your enterprise. Having worked on a number of large enterprise projects where upgrading the "legacy" system was the goal, I know full well that management are likely to prefer the monkey patches (i.e. Scala where the pain of rewriting components in a PoC way doesn't exceed the pain of threading your existing code at the bottlenecks), especially when the middle/technical managers do not highlight the drawbacks of the "solution" adequately. Three to nine months later the management (especially middle management) are hurting big time, but it is not usually the most appropriate time to do your "I told you so dance". Trust me!:) The the monkey patch isn't an ideal solution when you are simply trading one complexity of equal of greater value for the other and in 2-5 years the likelihood that there will be many more skilled engineers and architects that truly understand and have solid production experience in a truly concurrent language like Erlang is high in my estimation, so making a move to such a language/environment would not be as risky at that time. However, for 2-5 years you have been getting little out of your hardware and software together unless you overbuy hardware. In fact, in the case where legacy code already exists, there is never really an ideal solution. However, starting out with meaningful PoCs and structuring low risk exit points after conducting objective evaluations of these projects is key to the success of every "legacy solution". If in doubt and your legacy system isn't showing signs of hurting yet, you might want to wait and beef up on hardware. This is a transition period and if you are in a conservative IT enterprise this might be a better near-term approach.

Startups

Now for the startups or those starting afresh, there is mixed news: very good news and not-so-bad news. Green field projects, in theory, are always the best. You can get everything right from day one....well at least you can hope to do so (let us pretend for this article). My advise for your team is to take the plunge to Erlang. It is actually relatively low risk to do so for the following reasons:
  • It is a mature language and environment already tested by large and complex real world systems at telcos where reliability and scalability were and are top priorities.
  • There are a few good books out there on that will help you convert to Erlang and Functional Programming (see resources section below), although documentation and education on Erlang, generally, is somewhat lacking.
  • A framework for services has been designed, developed and put into production called OTP, which stands for the Open Telephony Platform and actually nothing specific to Telephony applications in it, it was simply developed by a Telco.
Erlang's approach to concurrency is interesting and takes some getting used to as it is a pure functional programming language unlike Scala which is a hybrid. Now the bad news is that there are very few people that know Erlang (or other pure concurrent languages), you need to hire at least one specialist with Erlang experience to guide your team to avoid the potholes and perhaps even stop you before you get to the edge of a cliff. They might not be cheap, but starting out it will mitigate project risk significantly. Another negative on the Erlang side of the argument is that you will not be able to use your Java classes straight from within Erlang (in a meaningful way - I'll cover this in the next part). The positive about this negative though is that you can encapsulate your Java code in an SOA way so that you can decouple and declutter subsystems within your enterprise, which is an important task to do regardless of which legacy solution you choose. Finally, good luck and let me know how it goes!

Resources

Programming Erlang: Software for a Concurrent World by Joe Armstrong

This book, for the moment, is the definitive guide for Erlang. If you have already decided to go the Erlang route then you will not mind (or perhaps not even notice) the "selling" of Erlang throughout the book. Overall this is an essential book to anyone learning Erlang at this point.

Real World Haskell

While Haskell isn't either Scala or Erlang, if you do not come from a functional programming frame of mind you will need a good book to introduce you gently into the realm of functional programming. This book does a pretty good job. While the syntax for Haskell is different to both Erlang and Scala, the hardest part of moving to functional programming isn't the syntax. Trust me on this one:)

Also consider me, Susan Potter, a resource if you need an Erlang software engineer with high-level enterprise architecture background in Java/J2EE/JEE and Ruby/Rails production environments. I am developing a risk management product in Erlang right now and I am available as a consultant through my consulting firm Finsignia, which mostly works with financial services firms such as investment banks and hedge funds, which is where my business domain expertise lies, but also works on designing, building and deploying reliable and robust server-side solutions for any industry.

CUBarCamp: Champaign-Urbana's Tech Meetup Update

Written on 8:13:00 PM by S. Potter

This is just an update blog post to let geeks in and around the Champaign-Urbana area know that the CU BarCamp is now scheduled for April 25th, 2009. The meetup will start at 2pm CDT and will be located at the Collective Turf Coworking space:

110 W. Main Street Urbana, IL 61801
It is right between the Heartland Gallery and Priceless Books and opposite the Urbana Business Association office and Crane Alley.

Want to participate?

If you want to volunteer, present or demo any topics at this BarCamp please update the CUBarCamp wiki page and join the Google Group for the CU BarCamp event. Hope to see you there!

YUI MenuBar: A Facebook-like Application Bar

Written on 6:59:00 PM by S. Potter

Below is a snippet of HTML code with embedded Javascript code that will create a top anchored menu bar very similar to what you see at the bottom of the Facebook page:

<!-- Add the following to your HTML/HEAD -->
<link rel="stylesheet" type="text/css" href="http://yui.yahooapis.com/2.7.0/build/fonts/fonts-min.css">
<link rel="stylesheet" type="text/css" href="http://yui.yahooapis.com/2.7.0/build/menu/assets/skins/sam/menu.css">
<script type="text/javascript" src="http://yui.yahooapis.com/2.7.0/build/yahoo-dom-event/yahoo-dom-event.js"></script>
<script type="text/javascript" src="http://yui.yahooapis.com/2.7.0/build/container/container_core-min.js"></script>
<script type="text/javascript" src="http://yui.yahooapis.com/2.7.0/build/menu/menu-min.js"></script>
<script type="text/javascript">
YAHOO.util.Event.onDOMReady(function () {
  var oMenuBar = new YAHOO.widget.MenuBar("menubar");
  oMenuBar.render();
});
</script>

<!-- in your HTML/BODY modify the following for your own "Facebook Application Bar" menu -->
<div id="menubar" class="yuimenubar">
  <div class="bd">
    <ul class="first-of-type">
      <li class="yuimenubaritem first-of-type">
        <a class="yuimenubaritemlabel" href="#file">Applications</a>
        <div id="file" class="yuimenu">
          <div class="bd">
            <ul>
              <li class="yuimenuitem">
                <a class="yuimenuitemlabel" href="#">Notes<em class="helptext">Ctrl + N</em></a>
              </li>
              <li class="yuimenuitem">
                <a class="yuimenuitemlabel" href="#">Links<em class="helptext">Ctrl + L</em></a>
              </li>
              <li class="yuimenuitem">
                <a class="yuimenuitemlabel" href="#open">Photos<em class="helptext">Ctrl + P</em></a>
                <div id="open" class="yuimenu">
                  <div class="bd">
                      <ul class="first-of-type">
                        <li class="yuimenuitem">
                            <a class="yuimenuitemlabel" href="#">Upload Album</a>
                        </li>
                        <li class="yuimenuitem">
                            <a class="yuimenuitemlabel" href="#">Browse Albums</a>
                        </li>
                      </ul>            
                  </div>
                </div>            
              </li>
            </ul>
          </div>
        </div>
      </li>
      <li class="yuimenubaritem first-of-type">
        <a class="yuimenubaritemlabel" href="#"edit>Something Else</a>
        <div id="pim" class="yuimenu">
          <div class="bd">
            <ul class="first-of-type">
              <li class="yuimenuitem">
                <a class="yuimenuitemlabel" href="#"> Bla 1</a>
              </li>
              <li class="yuimenuitem">
                <a class="yuimenuitemlabel" href="#"> Bla 2</a>
              </li>
            </ul>            
          </div>
        </div>
      </li>
    </ul>
  </div>
</div>
For further details about customizing the YUI MenuBar look and feel you should visit the API documentation

CUBarCamp: Champaign-Urbana's Tech Meetup 4/22 (tentatively)

Written on 12:04:00 AM by S. Potter

Just a quick post to mention that with the assistance of Brian Marick and hopefully a few more people in the weeks to come, BarCamp is coming to the Champaign-Urbana area and is to be held at the Collective Turf Coworking space in downtown Urbana (110 W. Main Street, Urbana, IL - opposite Crane Alley and above Priceless Books). To keep updated on the latest developments and participate in the event organizing join the CUBarCamp Google Group. Hope to see you there!

Coworking Space in Champaign Urbana (Illinois)

Written on 3:26:00 PM by S. Potter

Tweet from @SusanPotter about starting a Champaign-Urbana Coworking environment/venue Since I moved to Champaign-Urbana, IL (aka CU) in 2005 I have been looking for work space that would provide me with a more productive environment and social yet professional outlet to work in since I work primarily on telecommute contracts and lack face time with clients. Today I decided that I will setup a local coworking space for solo entrepreneurs and freelancers in and around the Champaign-Urbana area to fulfill my needs as a consultant and fulfill my new venture urge of late. Essentially coworking is defined as the following:

Coworking is a movement to create cafe-like community/collaboration spaces for developers, writers and independents.
This is exactly how I imagined my dream work space since I work a lot in coffee shops all around town throughout the work week and even during the weekend sometimes and find it surprisingly (initially) productive. I have seen these types of office spaces in larger metro areas and the terminology used tends to be different from place to place. Some terms used to mean a community of independent professionals sharing office space, facilities and amenities that I have come across on my travels include:
  • Coworking or Co-working (obviously)
  • Collaborative work environments (not used in the context of a company job description:))
  • Work Space Cooperative (this might be organized at the top levels slightly differently though)
  • Shared Work Space
  • Serviced Office Space
There are probably some slight variants on the definitions of these as compared to that of coworking, so some places will have more structure and a more formal setting, whereas coworking itself strives to create a more creative and collaborative environment for independent professionals. If you are in the CU (East Central Illinois) area and would be interested in working in such a coworking space, please drop me a line at any of the following locations: Are you a freelancer or entrepreneur in another area or region that is part of a coworking environment experience there? What do you like about your coworking space and why? What would you change about it if you could?

GEdit Extensions Project

Written on 11:54:00 PM by S. Potter

Just a quick post from a long blogging hiatus to let people know that I launched gedit-extensions where I will maintain a set of commonly used GEdit snippets and eventually plugins and tools that others might find useful. You can view the snippet source at the GitHub gedit-extensions repository. I plan on adding documentation to the gedit-extensions wiki in the next couple of days, but for now see the sneak peak in the Ruby example snippets at the end of the post. The name in the comments directly above the example snippet is the shortcut you can use. Type the first few characters of the shortcut and then Ctl+Space key combination to pull up all shortcuts the characters match so far. Choose the relevant shortcut from the drop down list. If the snippet contains variables the first variable will be highlighted for your to fill in, then press TAB to navigate to the next variable to fill in. As you will see most of the shortcuts so far are Ruby, Rails, RSpec or Cucumber specific since there are a few Python snippets I need to untangle from our in-house code references.

# shebang
#!/usr/bin/env ruby
# absrequire
require(File.join(File.dirname(__FILE__), "config", "enviornment")

# habtm
has_and_belongs_to_many :idiots
# unique
:unique => true
# habtmthru
has_and_belongs_to_many :idiots, :through => :village

# belongsto
belongs_to :village

# createtable
create_table :sessions do |t|
  t.column(:name, :string, :limit => 34)
  t.column(:description, :string, :limit => 255)
end

# addindex
add_index(:sessions, :name)
# changecolumn
change_column(:sessions, :name, :limit => 70)

# removecolumn
remove_column(:sessions, :name)
# irreversible
raise ActiveRecord::IrreversibleMigration, "the reason the migration is irreversible"

# resource
class PersonResource < ActiveResource::Base
  self.site = "http://domain.com:8080/"
  self.element_name = "contacts"
  self.prefix = "v1_0"
  self.user = "supo"
  self.password = "geditrocks"
  self.timeout = 60
end

# describe
describe PersonResource do
  before(:each) do
    @resource = mock(PersonResource)
  end

  it "should bla" do
    
  end

  after(:each) do
    @resource = nil
  end
end

# specmodelerror
it "should have an error on name" do
  @user.should have(1).errors_on(:name)
end

# specmodelrecords
it "should have 50 records" do
  @states.should have(50).records
end

# spectemplaterendered
it "should render 'common/user' on get '/user'" do
  get('/user')
  response.should(render_template("common/user"))
end

# specresponsetext
it "should return text 'You were signed out.' in response to post on '/signout'" do
  post('/signout')
  response.should have_text("You were signed out.")
end

# specresponseredirect
it "should redirect to action 'index' on post '/users'" do
  post('/users')
  response.should redirect_to(:action => 'index')
end

# specroutefor
it "should map #index to '/users'" do
  route_for(:controller => "users", :action => "index").should == "/users"
end

# specparamsfrom
it "should generate params for #update" do
  params_from(:put, "/users/1").should == {:controller => "users", :action => "update", :id => "1"}
end

# spectagexists
response.should have_tag('ul.menu#sidebar')

# specnestedtagexists
response.should have_tag('ul.menu') do
  with_tag('li', 'my menu item')
  # specchildtagexists
  with_tag('li', 'my second menu item')
end

# specblocktagexists
response[:sidebar].should have_tag('ul#sidebar')

# mockviewhelper
template.should_receive(:curent_user).and_return(mock(User))

# specsuccess
response.should be_success

# specredirect
response.should be_redirect

# feature
Feature: Search Jobs
  In order to find relevant jobs
  As a Job Seeker
  I need to find jobs relevant to me

  Scenario: Location Search
    Given a search form
    When I enter my zip code
    Then I should see all jobs in and around my zip code
  # scenario
  Scenario: Skills Search
    Given a search form
    When I enter skill keywords
    Then I should see all job descriptions that mention those keywords