The Scala vs Erlang Debate, Part 2: The Geek Off

Written on 3:47:00 PM by S. Potter

In the first part of The Scala vs Erlang Debate article series, I discussed various ways to go about adopting either Scala or Erlang and in what scenarios it made sense to go with one over the other from the manager's perspective. In summary the major advantage of Scala is that it runs on the JVM so it can access pre-existing Java libraries your enterprise has already developed. This is also Scala's major disadvantage because it makes a rather large assumption (which in my opinion would be incorrect the majority of the time) that the Java libraries being used are thread-safe. Erlang on the other hand doesn't allow you to integrate in the same OS process with Java libraries, because Erlang was built from the ground up with scalability and fault-tolerence in mind and ideally you shouldn't put Java into the run-time stack because it compromises Erlang's fault-tolerance features significantly. Instead the preferred approach when writing the biggest bottlenecks of your system in Erlang and integrating with pre-existing Java libraries/system is to use SOA to create separate, but integrated systems that are easy to swap out and upgrade at a later point. In my view, you should be refactoring your enterprise architecture by defining services anyway, since this will decouple systems and remove a great deal of headaches in IT maintenance and deployment. Today I wanted to geek out and differentiate the two environments on a more technical level for the middle/technical managers.

Scala: Technical Overview

Scala is an objected oriented, statically typed language with many functional programming capabilities built-in. However, it must be noted (because few Scala documents do this) that Scala is hybrid actor programming language and therefore a number of assumptions cannot be made. So by being a hybrid functional language it losses some of the benefits. As well as it's Java run-time support on the JVM it is also supported in Microsoft's Common Language Runtime (CLR) so .NET integration, in theory, is a breeze. Again this also assumes that the .NET libraries you use from Scala are thread-safe. On a related topic, .NET also has F# available, which is Microsoft's attempt at a hybrid functional programming language also. I have no comment on F# myself, since I have zero experience with it. Please chime in on the comments section if you have experience with F# and are able to contrast it with Scala.

Erlang: Technical Overview

Erlang on the other hand is a dynamic, purer functional programming language that supports light-weight processes in its own run-time environment (separate from OS processes), which send messages to each other asynchronously. One major side effect of being purer in its actor treatment is that there is virtually not state. There is no way to pass "state" around (a couple of minor exceptions), but there is a lambda calculus optimization that can be taken advantage of that will ease away our troubles called tail-recursion. Tail-recursion only partially works in Scala. For many developers that only have experience in procedural and Object-Oriented (OO) programming languages like Java, Python, Ruby, C++, C, Perl, C# this sounds completely insane. Two years ago I may have agreed with you, but today life is much simpler without state in my Erlang travels than when I had to write thread-safe Java code and I wish I had discovered this concept earlier. I will not pretend this notion didn't take some getting used to but today I see things a lot clearer and I am thankful for that clarity.

Scala: Concurrency

Concurrency in Scala is modeled using a concept of actors, except there are two types of actors: thread-based and event-based actors. Since Scala was implemented on top of the JVM, which is stack-based, Scala had to compromise on Erlang's one actor approach. This is because native threads, which the JVM uses for concurrency support, are much heavier-weight than Erlang's lightweight processes thus reducing the number of thread-based actors that could be launched inside one JVM. So Scala created this concept of a very peared down event-based actor, which does not have all the functionality of thread-based actors but does not incur the penalty of needing to be backed by a native thread. To be as complete as I can be in this section I should also note that Scala treats everything as an object, even though it has these two types of actors. Actors in Scala are just objects. In turn this may create duplication between objects and actors.

Erlang: Concurrency

The Erlang VM was built from the bottom up with lightweight processes (a.k.a. actors), therefore it is stackless and much better at lightweight concurrency. Erlang also has a built-in mechanism for load balancing these lightweight processes across all available CPU cores. This means Erlang has a massive advantage over JVM based functional programming languages at getting the most out of multi-core hardware, which is at the crux of the problem domain that software needs to address going forward.

Scala vs. Erlang: Performance

In terms of linear performance (sequential) Scala has been shown to beat Erlang in certain cases. However, concurrent performance consistently shows Erlang beating Scala, sometimes by wide margins on multi-core hardware. Not surprising considering what we learned in the concurrency sections above. When considering the problem we are trying to overcome - optimizing software on multi-core hardware - it seems clear which of these benchmarks are more important! :)

Conclusion

I think it is clear, which side of this debate I firmly stand, but I have tried to demonstrate both good and bad traits of each programming language from a technical perspective that might translate into a business case or reason for choosing one over the other in your organization. If I have time I might do a part 3, which will talk a little about syntax and DSL support. This might round out the argument since it could be argued that Scala wins in the third part by a hair, but all is not lost for Erlang if the underlying problem you need to solve is related to optimizing software for modern hardware and building robust systems at the same time. My vote still goes to Erlang despite the lack of built-in features that make designing DSLs easier, but it is not difficult to build your own DSL engine in Erlang either.

Limitations of this series:

There are quite a few topics I left out of this post that might be of interest for some applications or services. Namely:
  • Hot swapping: Erlang has great hot swapping capabilities for production environments that minimizes or possibly eliminates downtime of systems. This feature in Scala is not close in capability or reliability at this point.
  • Garbage collection: without getting into too many details Erlang probably wins this section by a hair (not significant to write home about). Long sweeps will not be a problem in Erlang and only a problem in JVM languages on rare occasions.
  • Distributed nodes: The clear winner here would be Erlang because built-in to the environment are all the multi-node features that make distributed systems a breeze. Scala on the other hand has different frameworks and libraries to facilitate distributing nodes of logic on different hosts.
  • [Added Nov-2009] Frameworks: As far as I know, Scala doesn't have an OTP killer, not even a contender to compete with Erlang's Open Telephony Platform (OTP). Sure it will take some time, even Haskell, which has been around much longer than Scala is slightly jealous of OTP (I saw the way Don Stewart looked at me when I asked him if Haskell had framework similar to OTP;) ). Supervision trees along make it stand out as the best thing since sliced bread, at least for this particular software engineer.

Resources:

Corrections (added February 2010):

  • Erlang is not a pure functional language. However, it does not have a polluted (impure) treatment of actors like Scala:) Some think this is a good-enough approach to be able to leverage the JVM, when running on the JVM is really important, perhaps 4 times out of 5 it might be "good enough". I just wonder if it really matters for most projects that you run on the JVM!?
  • "Scala's lack of tail recursion is entirely due to a trade-off. The JVM doesn't do tail recursion itself. It is possible to add tail recursion on top of the JVM using various tricks, but they are expensive. SISC does its own stack based heap and Kawa uses trampolining." -- James Iry

If you enjoyed this post Subscribe to our feed

26 Comments

  1. James Iry |

    Erlang is not a purely functional programming language. A purely functional programming language must separate IO, mutation, and other side effects from pure computation. In Erlang you can interleave side effects anywhere. That puts Erlang in the impure functional languages camp along with Scheme, Common Lisp, SML, OCaml, and yes, Scala. In particular, note that Erlang or Scala actor message sends and receives are side effects.

    For purely functional programming languages see Haskell, Clean, and Miranda. All of them make a very firm distinction between pure computation and things with side effects.

     
  2. James Iry |

    Another comment. The reason Scala doesn't have full tail call optimization has nothing to do with its impurity. Scheme, SML, OCaml, and Erlang are all impure but have tail recursion.

    Scala's lack of tail recursion is entirely due to a trade-off. The JVM doesn't do tail recursion itself. It is possible to add tail recursion on top of the JVM using various tricks, but they are expensive. SISC does its own stack based heap and Kawa uses trampolining.

    An interesting compromise, available in Clojure now and Scala soon, is to let programmers manually specify where they want to do tail calls via trampolining.

     
  3. S. Potter |

    @James I will clarify what I meant:
    In an inter-process (or inter-actor) frame of reference, you are absolutely correct - Erlang is impure! I should have caught this in my proof-reading, but since I am terrible at both writing and proof-reading it was inevitable that I missed this (and probably quite a few others).

    However, within the same process/actor there are no side effects in Erlang. Scala allows side effects within a thread/actor. Erlang and Scala are fundamentally different in this regard. And it is this distinction I was referring to throughout the posting.

    Thanks for keeping me honest!

     
  4. S. Potter |

    @James I thought I had mentioned that the lack of full tail-recursion support in Scala was due to the JVM not being stackless like the Erlang VM? I will reread tomorrow morning to find the offending part of the posting.

     
  5. Jason R |

    Jason: I think actually the more important point she might be making is between hybrid vs non-hybrid functional languages. Scala tries to marry up a quadrillion things up in one language, whereas Erlang is simply functional.

    Whether it is pure or not, I don't think that was the point...it is more a matter of hybrid-ness NOT purity.

    Having a (one-sided) academic debate about definitions of functional language purity seems rather pointless.

     
  6. Jason R |

    Apparently I misread your name...sorry James.

     
  7. momingle |

    I started learning both Scala and Erlang the same time. I started with Scala first, went through all the tutorial and guides. Then I went to erlang. From language perspective, I found erlang's syntax much cleaner. That is probably due to Scala is a static type language and erlang is a dynamic type language and Scala has to be both OO and FP the same time. But even the FP syntax in Scala seems a bit overkill to me. Pattern matching is a lot easier in erlang than using case class in Scala. That is just my beginner's opinion. I am no language expert. I just voice my opinion as a developer who wants to get things done and quick. That means I want to be able to learn a language fast. Syntax does matter. One of the reasons so many developers were able to switch to Ruby quickly is due to its simplicity of its syntax.

     
  8. Anonymous |

    Would someone please mention execution speed? My impression from reading around is that Scala is not much worse than Java whereas Erlang is a whole lot worse. Execution speed may have more to do with tooling than language design, but it's certainly a factor in deciding on a language.

     
  9. Matt |

    "However, within the same process/actor there are no side effects in Erlang."

    The process dictionary?

    Also, I'd argue that trying to make a distinction about purity "within the same process" is largely moot: if I call io:format(), it produces a side effect. It makes no practical difference if it's implemented by sending a message to another process that performs the side effect, or whether the current process performs the side effect. You get no more purity than if you'd called println in Scala.

    "I thought I had mentioned that the lack of full tail-recursion support in Scala was due to the JVM not being stackless like the Erlang VM" -- the Erlang VM uses a stack, doesn't it?

     
  10. Emil Ivanov |

    "... [Scala] makes a rather large assumption [...] that the Java libraries being used are thread-safe."

    This is not correct.
    It just doesn't. Actually - it's the other way around - Scala actors allow you to run non thread-safe code in parallel. The code inside of the actor doesn't need to be thread-safe. That is one of the advantages of using actors over threads with locks.

     
  11. Paul Prescod |

    Jason: James was simply correcting facts in the article. It isn't a "debate". It's just getting mistakes fixed. I noticed the exact same things in my reading. 1. Erlang is not purely functional and 2. there is no relationship between being purely functional and implementing TCO.

    Momingle: Ruby does not have a simple syntax. It is much more complicated than (for example) Python or Javascript, to say nothing of Lisp or Smalltalk (which are even simpler). For example, Ruby has begin/end and curly braces. If statements can go at the beginning or end of lines. It overloads the "<<" operator to mean a bunch of different things. There are a ton of different kinds of attributes (instance attributes, class attributes, class instance attributes, maybe more). It has different syntaxes for blocks versus methods versus (lambda) functions, each with unique syntax, when Javascript, Lisp and Python get along with just functions.

     
  12. puzza007 |

    "One major side effect of being a pure functional programming language is that there is NO STATE. There is no way to pass "state" around, but there is a lambda calculus optimization that can be taken advantage of that will ease away our troubles called tail-recursion."

    I think what you're trying to say here is that there's no mutable state/variables in Erlang. To achieve a state update you'd typically call your recursive loop function with a new variable representing the updated state. Tail-call optimisation makes sure that this can be done with a constant stack size.

     
  13. AndrewMcV |

    erlang is quite a bit slower than scala for sequential stuff. http://shootout.alioth.debian.org/u32q/benchmark.php?test=all&lang=scala&lang2=hipe&box=1

     
  14. S. Potter |

    Actually I see a very different story in the results.

    The bigger issue with the results seems to show that Scala really isn't scalable in resource usage and for longer running computations is outperformed by Erlang. Also I noticed how the benchmark tests are running an older Erlang before the new SMP scheduler was introduced.

    Based on the benchmarks I would be very worried about memory availability for Scala systems that were more than trivial. It looks very worrying to me actually.

    The big question is why anyone is still only interested in sequential (linear) performance when parallel performance is clearly what matters in a multi-core age.

    Take a look at this concurrent benchmark to look at real world concurrent results:
    http://shootout.alioth.debian.org/u32q/benchmark.php?test=threadring&lang=all&box=1

    Scala / Erlang running time ratio = 2.921866217

    Note: the above benchmark used an older Erlang VM without the new SMP scheduler which is likely to vastly improve on this performance. Thus the ratio is likely to be even more embarrassing for Scala.

     
  15. puzza007 |

    Why didn't you allow my comment earlier?

     
  16. S. Potter |

    @puzza007 I haven't withheld any comments on this post yet. When did you make the comment? I don't see it in blogger.

     
  17. Anonymous |

    Losing a single benchmark is hardly what I would call embarrassing for Scala, a more general purpose language. It would be interesting to see how Erlang pays off in the real world right now, aside from benchmarks. How many and what types of programs are written for multiple cpus and are just waiting to be rewritten in Erlang so that they execute more quickly? Would programs written in Erlang be faster on current desktops? How many CPUs do you need to see Erlang do things more quickly?

     
  18. S. Potter |

    I find it interesting that it is a "single" benchmark when Erlang wins, but not when Scala wins.

    Also my blog posts about Erlang vs Scala are specifically talking about multi-core applications (on servers primarily). I have not suggested that Erlang is a multi-purpose programming language for desktop applications. However, at the same time I do not believe Scala fits the multi-purpose programming language that would be particularly suitable for desktop applications either. I am a little confused by your argument?

    I have made no such claims about multi-purposeness, nor do I believe Scala fits that profile either. So the comment seems a little out of context.

    Perhaps you could expand if I am missing your point?

     
  19. S. Potter |

    In addition I wish to point out that Erlang has been used in large scale production systems for over 15 years already. I think it's real world usage and maturity far exceeds Scala's currently.

    I hardly think that was a good argument for Scala in comparison to Erlang!

    I do wish to clarify my opinions though. I personally much prefer using Erlang over Scala for multiple reasons! However, despite Scala's current relative immaturity (which will not be so much of an issue after a few years if it continues with it's current momentum) it has potential. I don't really know if I would be swayed by Scala at that point, because I can't predict the future, but I do leave my mind open to that possibility.

    I do enjoy seeing Scala's fan boys getting frazzled by these benchmarks though:)

    Peace!

     
  20. S. Potter |

    @puzza007 you were right, I had to dig around in blogger to find it. I didn't receive the email notification for this and a few other comments. I have no accepted them all now.

    Sorry for any confusion. I will be responding next week regarding some comments.

     
  21. S. Potter |

    s/I have no accepted/I have NOW accepted/

     
  22. Jayden Cope |

    Paul Prescod: I don't think Susan ever said facts were debatable.

    It is interesting that you (Paul) are implying there cannot be any debate in terms of which programming language you want to use. Apparently you think Scala is the only language anyone should use?

    Good to know, Paul.

     
  23. S. Potter |

    Jayden: best not misquote him either;)

    Emil Ivanov: Actually I did mention the two types of actors in Scala, please see:
    "So Scala created this concept of a very peared down event-based actor, which does not have all the functionality of thread-based actors but does not incur the penalty of needing to be backed by a native thread."

    Where is the error? If you have a lot of thread-based actors you are in trouble in Scala. If your app needs a lot of thread-based actors then you aren't going to be able to scale as easily as with event-based actors which are much more limiting. This *might* be a benefit in your applications, but if you don't change the way you write your software to be more event-based then you still have a scaling issue.

    Paul Prescod: Let me respond to this comment since I have already responded to James' comment you mentioned: "2. there is no relationship between being purely functional and implementing TCO."
    This is definitely something I want to clarify in a future post, but I do see a correlation between the fact that Scala actively encourages mutability with thread-based actors that need to consider thread-safety as an extra thought exercise in development, thread-based actors are less scalable, heavier weight (thus you need to throw more hardware are the problem) and debugging, troubleshooting costs go way up due to this complexity. There are some advantages to this also, namely legacy support. This is a very practical reason to do this in Scala. If you aren't concerned with legacy, I see the Scala practical impurity (not just theoretical impurity) as problematic and adding cycles to development as well as hardware to your production environments.

    I commented earlier on the "purity" comment. Yes, Erlang is not a pure functional programming language in an academic sense. It is practically pure. It does have a process map that you can modifying interally and some I/O actions do have consequences, but these were added for practicality, not to encourage mutability in your software writing. The programming style encouraged by Erlang is pure functional in nature, NOT a messy hybrid that Scala actively encourages.

    This is a fundamental difference and pretending it doesn't exist because of an academic definition is like an ostrich digging its head in the sand.

    I have already mentioned that there are some positives as well as negatives to this.

    There are practical limitations that Erlang has. Perhaps I didn't do enough of a job of highlighting these although I did mention them, so that might have been a very valid point. Highlight those practical limitations. This blog post is for people that build real world production systems NOT academics discussing technicalities.

    And lastly I only used "debate" in the sense that it is a debate as to which language is right for your needs.

    We can all have different opinions on the cost of a hybrid approach in relation to TCO, but I will not be changing mine just to sweep Scala's limitations (as I see them) under the carpet.

    Hope that helps clarify a little.
    Susan

     
  24. Matt |

    An argument you made in your article was, "Scala is not a pure functional programming language and therefore many corollaries from lambda calculus cannot be applied for optimization purposes", in particular, in Erlang, you argue "there is a lambda calculus optimization that can be taken advantage of that will ease away our troubles called tail-recursion."

    When this is pointed at as a flawed argument, you replied, rather than admitting a mistake, by "clarifying", and dismissing factual criticism as irrelevant concerns from "academic" vs "real-world" programmers. The truth is, if you invoke academic concepts like "corollaries from lambda calculus" and try and draw real-world conclusions about it, it's reasonable to criticise the validity or otherwise of your argument, and it is certainly not "like an ostrich digging its head in the sand" to do so.

     
  25. S. Potter |

    Matt: Actually I readily admitted my factual mistake with "you are absolutely correct - Erlang is impure!". I suggest you read the all the comments. What did I say that was dismissive of that particular fact?

    Other "corrections" in the comments have argued that Scala is just as impure as Erlang. My argument is that there is still a difference in the degree to which Scala promotes a certain style of programming that is not functional in nature.

    I am happy to discuss that, but let us separate what I did say to what you said I said because they are two different things.

     
  26. ddadmin |

    Scala is a hybrid Object-Oriented/Functional Programming language on the JVM. When I heard that Twitter was using Scala, I was curious and started collecting all the sites and articles to learn scala programming. If you are interested check the link below for the big list I have gathered (more than 200 sites) for learning scala programming.
    http://markthispage.blogspot.com/2009/06/more-than-100-sites-to-study-scala.html

     

Post a Comment