Category Archives: Java

Clojure, Programming, Scala

Horses for courses: choosing Scala or Clojure

So one of the questions after my recent talk trying to compare Scala and Clojure (something that I suspect is going to be an ongoing project as I hone the message and the tone) was about whether the languages had problem domains they were more suited too. That’s an interesting question because I think they do and I thought I might be interesting to go through some of the decision making process in a more considered fashion than answering questions after a talk allows you to do.

So some of the obvious applications are that if you want to leverage some Java frameworks and infrastructure then you definitely want to use Scala. Things like JPA, Spring-injection, Hibernate and bean-reflection are a lot easier with Scala; in Clojure you tend to be dancing around the expectations these frameworks have that they are working with concrete bean-like entities.

If you are going to work with concurrency or flexible data formats like CSV and JSON I think you definitely want to be using Clojure. Clojure has good multi-core concurrency that is pretty invisible to you as a programmer. The key thing is avoiding functions with side effects and making sure you update dependent state in a single function (transaction). After that you can rely on the language and its attendant frameworks to provide a lot of powerful concurrency.

Similarly LISP syntax and flexible data go hand in hand so writing powerful data transforms seems second nature because you are using fundamental concepts in the language syntax.

Algorithm and closed-domain problems are interesting. My personal view is that I find recursion easier in Clojure due to things like the explicit recur function and the support for variable-arity function definitions. Clojure’s default lazy sequences also make it easier to explore very large problem spaces. On the other hand if you have problems that can be expressed by state machines or transitions then you might be able to express the solution to a problem very effectively in a Scala case class hierarchy.

When it comes to exploring the capabilities of Java libraries I tend to use the Scala console but for general programming (slide code examples, exploratory programming) I do tend to find myself spending more time in LightTable‘s Instarepl.

When it comes to datastore programming both languages are actually pretty clunky because they devolve handling this down to various third-party libraries. Clojure does pretty well with document databases and key-value stores. Scala is great for interacting with the AWS Java libraries and neither deals particularly well with relational data.

For web programming neither is brilliant but Scala definitely has the edge in terms of mature and full-featured web frameworks. Clojure is definitely more in the log cabin phase of framework support currently.

Standard
Java

Hibernate temporary tables

I was surprised to discover that Hibernate makes use of temporary tables when performing deletes. The creation of the table was triggering an alert in a database monitor for an application. The monitor logged the creation of tables beginning with the prefix HT_ and a name that mirrored that of an entity tablename.

Initially I must admit that I was thrown very far off track and started looking through the schema creation scripts and codebase looking for something, even checking HT as initials against the developer names. Naturally we tried deleting the tables and soon enough they were recreated.

As with a lot of modern programming bugs the answer came through some frustrated Googling and the realisation that HT might stand for Hibernate Table. With that insight in place I was then able to find this excellent explanation of the situation and I get another self-righteous data point in my dislike of ORM.

This temporary table is required for two reasons, firstly because ORM’s do not actually do a proper mapping between the domain concepts of a relational store and objects and secondly because Hibernate wants to create a verisimilitude around the concept of a class hierarchy.

I don’t think there is a better solution than what Hibernate is offering here but I do think it is the wrong problem being solved in a clever way.

Standard
Scala

What’s the problem with Scala’s “lazy” vals?

Scala has a value modifier called “lazy” and I have a bit of a problem with it. Let’s start with the name, lazy is used to indicate that the value is not defined when declared but when the value is read. Really it isn’t lazy at all it is actually a memoisation wrapper around the value.

Now that’s a pretty handy thing and there are definitely some use cases for this. In particular where the calculation of the value is truly invariant and expensive. Some people suggest that it can be used to cache the results of service or database lookups but that is clearly madness unless the query really does return an invariant.

However I think that there is a very subtle problem with the misapplication of lazy and that is that it introduces state into an otherwise pure function, namely whether the value has been calculated or not and what the value is when calculated. This state to me actually reduces the utility of a pure function in terms or reuse as the consumer has to be very aware in time of when the function is going to be invoked and what its value is going to be over time.

It’s rare that lazy over-application goes wrong because I mostly see it in short-lived instances. However that very lifespan seems to suggest that there is no real value in lazy evaluation over eager declaration. After all if the function never gets called then it doesn’t cost anything and true constants should be in the companion object. In long-lived objects though I think there is serious potential for lazy to go wrong and therefore it should be applied sparingly where a positive impact can be proved.

Standard
Clojure

London Clojure Maze solver dojo

Last month we had another team code competition, this time centered around writing code that trys to solve a maze. Clojure seems quite apt for creating these kind of challenges as it has a lot of support for dynamic code evaluation and the functional paradigm makes writing callbacks a lot easier.

Just like the Battleships dojo it was interesting in that the random strategy was a good local maximum. However one revalation that the maze wasn’t cyclic later then left-wall hugging was kicking everyone ass. That then left dead-end elimination as the only possible way to produce a faster solver. Which our team failed to do sadly. Right idea, wrong turning table.

We also got bogged down on a Clojure issue which has come up a few times at the dojo. I’ll summarise it here: should you be using Clojure 1.4? Your library syntax and server compatibility depends on the answer to this and there is no good error message that is going to tell you that the language syntax has changed.

The competitive dojo is an interesting environment where only the best work process and most pragmatic code can thrive. It is an interesting critique of hammock-style as the result of all thinking and beard-stroking better be order of magntiudes better than the obvious answer.

We also got to see a good example of beard-stroking abstraction this month with Chris Ford’s introduction to the theory of music and its abstractions in a general purpose computing language. An amazing talk which combined education with an amazing abstraction over music itself.

Standard
Clojure

Clojure: does a map contain all the specified keys?

An interesting problem came up this week during some Clojure batch work. How do you cleanly say that a map contains a set of given keys? The context is that during batch processing you only want to perform some operations if earlier operations have populated the right data set.

There’s probably some neat trick or built-in function but this is what I’ve come up with. I quite like the mapping of the count, it would look even better if I didn’t have to apply the equals but it’s not bad.

(defn has-keys? [m keys]
  (apply = (map count [keys (select-keys m keys)])))

(def map-data {:a 1 :b 3})

(has-keys? map-data [:a :c]) ; false

(has-keys? map-data [:a :b]) ; true
Standard
Clojure

EuroClojure Day 2

Okay so this post maybe happening a little later than Friday but in my defence there were some excellent conversations to go with the after-conference drinks.

Day 2 featured two talks by Rich Hickey, I had already seen some of the Datomic stuff from QCon and the web so I found the stuff on the new reducers library more engaging. I have never thought of map having an implicit ordering promise.

Meikel Brandmeyer gave a historical review of lazy seq which was really helpful for understanding laziness (something I have a bit of a problem with). One of the real highlights though was Chris Ford’s talk about canon music. It started with a good gag about sheet music being a DSL for using the finite state machine otherwise known as a musician. However the really amazing thing was Chris’s abstraction of the score and subsequent transformations of the abstract score to end up with variations on the base canon he had chosen. Really amazing. Chris’s talk really shouldn’t have been a lightning talk, it is about the only quibble I had with the programming.

Sam Newman also had an excellent closing line in his lightning talk on Riemann, which was if people want Clojure to be adopted widely then the secret is to create great things with Clojure.

Standard
Java

EuroClojure 2012 Day 1

So there were definitely two big themes in the talks on the first day of the conference.

The first has been about how to use event-based systems to create flexible aggregate data models. All speakers seem to have settled on a reduce or foldLeft approach for creating the aggregate but there have been two models put forward already CQRS and a kind of Aggregate query bus but really it seems that responsibility for accepting event data and allow querying and access to aggregated views seem to be responsibilities in the same system.

The other thing has been creating query systems using logical predicates. The were no less than three generic query systems put forward: core.logic for low-level flexible implementations that identify either data or results and two general query libraries: one from Datomic and the other from Cascalog.

Standard
Java

The dark side of the entrepeneur

The recent disgrace of Peter Cruddas is particularly interesting to me as a way of illustrating the dangers of bring the vaunted “skills” of the the private sector into public life. I actually feel sorry for Cruddas, I consulted at his company once and met him at a project conclusion meeting. When he talks about “bluster” and the fact that he was not in a position to deliver what he was claiming to be able to I actually believe him.

A lot of “enterprising” sorts (not just the ones born in the East End) are in the habit of “over-promising” what they can deliver. Less politely, a lot of them lie about their achievements, support and capabilities. Partly out of insecurity, like Cruddas, partly out of habit, partly out of ego. The thing is that in their natural environment they mix mostly with people who are doing the same thing and are never subject to any kind of meaningful scrutiny. Everyone can swim along in their happy bubble of hyperbole.

Stepping outside of that cosy world into the more brutal and unforgiving political one is fraught with dangers. Talking a load of shit to an investor or customer is unlikely to result in the conversation being plastered over the Internet. In fact often these conversations are private because of the collusion of the listeners, everyone is after something after all.

Entrepreneurs often regard journalists and politicians with scorn and they often find it hard to break the swaggering, boastful exaggerating habits of their private world. Working in public life requires a very different approach, despite the best efforts of vested interests public business is a lot more transparent than the private sector.

Private individuals have a lot to offer the state but I suspect that direct employment is the worst way to access those abilities. Contracting deliverable products would seem far more sensible. Advice is far better taken for the price of a coffee than a “premier league” donation.

Standard
Clojure, Programming

January’s London Clojure Dojo

January meant Battleships. More specifically battling battleships. Five teams created players and duked it out during the dojo with a tremendously narrow margin of victory. So what did we learn?

Well first of all randomly placing ships and shooting is actually a pretty good strategy. This is what the default player does and any deviation from it can be pretty badly punished by it.

One simple thing that people did to start improving over the random start was restricting placement of ships to a single half or quarter of the board. Doing this allowed most teams to start beating the initial strategy.

However clustering your ships is only effective against random shot placement so when people start implementing targeting you actually become more vulnerable. The first effective targeting strategy was surprisingly simple, if you hit something choose an adjacent square as your next target.

The team that squeezed to the top refined this by choosing an adjacent square that hadn’t already been fired at. The next level of improvement would probably be a non-trivial look at the probability that another ship square lay in the adjacent squares by looking at the information surrounding them.

There was a lot of work around the concepts of adjacency and whether the square had been fired at and the teams all seemed to converge towards the clojure.set library (if they were aware of it).

I’m now thinking of what fiendish problem would force and exploration of this library as it seems incredibly powerful for all different kinds of problems.

Standard
Java

Bottle on Epio

As WSGI-based framework you can get Bottle running on ep.io. However it isn’t part of the documentation as yet.

The basic setup is an app.py, requirements.txt and the epio.ini. Requirements obviously just has bottle.

Epio.ini

[wsgi]

entrypoint = app:app

requirements = requirements.txt

The app.py file should be:

import bottle

app = bottle.app()

@bottle.route('/')
def home():
  return {"message" : "Hello world"}

That should give you a basic JSON service running quickly.

Standard