Category Archives: Java

Java

Delta versus Chain

There are already two styles of persistence code in my daily work codebase. The first is based on the concept of generating the delta of the change you want to make. The second is the construction of the new data map by chaining functions together.

Let’s take some examples:

delta = { 'when' : now(), 'comment' : 'Hulk smash!', 'tags' : tags.append(new_tags) }
comment.update(delta)
store.save(comment)

store.save(update_timestamp(comment(update_tags(new_tags), 'Hulk smash!)))

Hopefully you can already see some of the pros and cons about both approaches. The chaining (or trainwreck style) ends up being very short and powerful. However when you need to pass in a few parameters at different points in the chain it can get very complicated to read.

The delta pattern allows you to accumulate changes into the delta neatly and to combine several changes (data, timestamps, audit entries) into one atomic write that is, in my view, easier to read than the chained example. However the boilerplate doesn’t go away and starts to get irritating after a while.

These are Python examples because in Clojure the threading macros and argument lists take away my comprehension concerns.

At the moment I’m continuing to experiment with both strategies and looking for a way to incorporate the best elements of both the comprehension of the delta and the conciseness of chaining.

Standard
Clojure

Why I’m finding Clojurescript underwhelming

I noticed Clojurescript in Github before the big announcement and thought it was an interesting idea. I am a big fan in general about having a Clojure syntax that compiles to Javascript. As a platform it is even more ubiquitous than Java and it would be a great way of simplifying Javascript’s closure and function syntax.

However in practice Clojurescript has been desperately disappointing for me. Firstly there is the weird decision to not have the code run on OpenJDK. This really limits its utility: I don’t seem to have a machine with a compatible setup at the moment despite having various flavours of Javascript interpreters available.

Then while looking for an answer as to how soon this problem is likely to be resolved I discovered this thread which was another level of disappointment. The original post is undiplomatic, perhaps even inflammatory, however the response indicates a level of befuddling clueless-ness.

If you want something to compile into Javascript I think you actually do want it to compile into good idiomatic Javascript unless you have a really good reason not to. You also do want to be able to use really good existing frameworks like jQuery (which really is the defacto standard right now).

The reason I think these are reasonable requests is that Coffeescript seems to manage to do both. Before Coffeescript maybe Clojurescript’s idiosyncrasies would have been forgiveable but being late to the party as well as being less well-mannered makes the defiance in the response seem poorly judged.

I am not sure what Clojurescript is really for (apparently it is aimed at a future community of people that don’t exist yet, which is … helpful). I don’t feel that it is really simpatico with the existing Javascript code that works in the browser and I am not sure it really has a place in the server-side world of Node.js where it might have been a better fit.

I remain open-minded though and would be willing to give Clojurescript a second go once the dust has settled a bit.

Update: I’ve written a follow up to this post

Standard
Clojure

Dojos: One conversation in the room not in the driving seats

Bruce Durling and I had a conversational meeting/talk last week about doing a Clojure dojo here in London for the last year. One of the attendees was Nicholas Tollervey the organiser of the Python dojo. In the course of discussing formats there was an interesting discussion about the concept of “one conversation”, which is to say that people stand a better chance of understanding something is there is just one conversation going where everyone is contributed to the shared understanding rather than several conversations where nothing may be getting understood. Nicholas had read the original Paris dojos and interpreted their rules as saying that the conversation is entirely owned by the pilot and co-pilot and that the audience cannot contribute.

My understanding (this may be a Brazilian addition to the Paris method) and the way I would practice dojos is that all the participants are in the conversation but that once a point or question has been raised by the audience then that issue must be addressed before the coding pair continue. So for example if an audience member doesn’t understand a piece of code or a function they should ask what it means and then the coding pair pause and an explanation is explored until the person asking the question feels they understand what is happening. The coding pair then resume.

With that said there are some important points of concern in terms of facilitation. Firstly the coding pair should be shielded from barracking, “Why are you doing it like that?” can be a good a question but is rarely helpful. A better question would be “Are you trying to solve the problem with method A? Would method B be better?”. But the best type of this question is not asking it, there may be a better way to do something but watching people explore an alternative route might be informative in its own right, even if it ultimately confirms your own views.

Questions should be “points of order”, ideally based on facts not opinion and aimed at clarifying understanding. Philosophical points of view are best expressed at natural break points in the coding or down the pub.

Once a question has been asked the focus of the conversation moves to the person asking the question. This often means that the person asking the question feels a tremendous amount of pressure to say that they understand something and allow the focus of conversation to move on. I know I have often been frustrated and embarrassed with myself for not getting a point. However if you don’t really understand something it is important to say so as it is likely that other people in the room feel the same way and that those trying to furnish an explanation have not done so satisfactorily and need to try again. Facilitators need to keep the safety up here if the group struggles to be supportive.

A final point is that every once in a while you get the really good question. The one that is either going to take a massive amount of effort to answer, that is fundamental to the problem or that lacks a definitive answer. Some people regard these questions as rabbit holes and while it is true that you can often kiss the coding goodbye for the rest of the night it is often these questions that lead to the most memorable moments of a dojo. I remember this happening when Ola Bini gave a spontaneous lecture on Ruby’s object lifecycle event hooks. It started with someone using an unusual technique and it ended up being a really enlightening trip through the guts of Ruby.

Dojos are primarily about learning and these side trips are as important as powering down the highway.

Standard
Scala

Using function passing to separate concerns

So I had a difficult problem today where I needed to access a service but I couldn’t inject it due to the service having slightly too many concerns and hence a circular dependency in the Dependency Injection. So there is right way of dealing with this, which is to refactor the concerns of the collaborator until the dependency goes away. Unfortunately there is also the incomplete user journey that is holding everyone up unless a quick hacky fix is found.

Enter function passing to the rescue! In my case we are talking about an MVC web application where the Model is generating the circular dependency but it is possible to inject the collaborator into the Controller. The trouble is that my code separates the concerns of the Controller and the Model such that the Controller asks the Model to perform an operation and the Model returns the result of executing the operation. This is clean to develop and read but it also means that the Controller cannot access any of the internal operations of the Model, which is what I need for the invocation of the collaborator.

For a moment I thought I had to break the encapsulation between the two and pass the internal information of the Model operation out with the operation result, however function passing is a more elegant solution that keeps my concerns separate. Essentially the Controller will ask the Model to do something and also provide a function for the Model to execute if it wishes to invoke the collaborator.

Let’s try to illustrate this in code


class Controller (stuff: StuffRepository, messages : MessageDispatcher) {

post("/handle/stuff/:stuffId") {

def messageCallback(id: StuffId, message : Message) { messages.send(Messages.StuffUpdated, message, Some(id)) }

stuff.storeChange(params("stuffId"), messageCallback) }}

class StuffRepository {

def storeChange(id : String, callback : (StuffId, Message) => Unit) = {

makeChanges(id) match {

case Success => callback(StuffId(id), Message("Changes completed")) ; Success

case Failure => Incomplete }}

Hopefully that’s all relatively obvious, you can also type the callback for clarity and make it an Option if you don’t always want the callback invoked. If you don’t like the closure you can probably re-write is as a function that partially applies MessageDispatcher and then returns the function.

Some of you are going to be thinking this is totally obvious (particularly if you a Javascript fiend) and that’s cool but I do think it is an interesting technique for keeping responsibilities separated without very complex patterns. It is also something that is only possible with proper first order functions.

Standard
Scala

Scala can be the replacement for Enterprise Java

Last week I had the chance to take a look at one of my worse fears about Scala: really bad Scala code. As I thought it might it did include a partial application, I was very grateful it didn’t abuse apply. It also managed to bring in a lot of classic bad behaviour like catching Exception and then swallowing it.

It has always been one of my fears that when someone went rogue with Scala the resulting mess would be so bad it would be a kind of code Chernobyl, its fearsome incomprehensibleness would be like the Lift code only without the brevity.

When I finally had to confront it I thought: this is actually no worse than really bad Java code.

I know that it could potentially be worse but actually it would take someone who was actually really good at Scala to construct the nightmare-ish worst case scenario I had imagined. In most cases terrible Scala code is going to look a lot like terrible Java code.

So that is really my last concern out of the way. We might as well use Scala for the good things it offers like immutability and first-order functions. The boost to productivity is going to outweigh the potential crap that might also be generated.

Standard
Programming, Scala

Mockito and Scala

Scala is a language that really cares about types and mocking is the art of passing one type off as another so it is not that surprising that the two don’t get along famously. It is also a bit off probably that we are using Java mocking frameworks with Scala code which means you sometimes need to know too much about how Scala creates its bytecode.

Here are two recent problems: the “ongoing stubbing” issue and optional parameters with defaults (which can generally be problematic anyway as they change the conventions of Scala function calling).

Ongoing stubbing is an error that appears when you want to mock untyped (i.e. Java 1.4) code. You can recognise it by the hateful “?” characters that appear in the error messages. Our example was wanting to mock the request parameters of Servlet 2.4. Now we all know that the request parameters (like everything else in a HTTP request) are Strings. But in Servlet 2.4 they are “?” or untyped. Servlet 2.5 is typed and the first thing I would say about an ongoing stubbing issue is to see if there is Java 1.5 compatible version of whatever it is you are trying to mock. If it is your own code, FFS, add generics now!

If it is a library that you have no control over (like Servlet) then I have some bad news, I don’t know of any way to get around this issue, Scala knows that the underlying code is unknown so even if you specify Strings in your mock code it won’t let it compile and if you don’t specify a type your code still won’t compile. In the end I created a Stub sub-class of HttpServletRequest that returned String types (which is exactly what happens at runtime, thank you very much Scala compiler).

Okay so optional parameters in mocked code? So I love named parameters and default values because I think they are probably 100% (no, perhaps 175%) better at communicating a valid operating state for the code than dependency injection. However when you want to mock and set an expectation on a Scala function that uses a default value you will often get an error message saying that the mock was not called or was not called with the correct parameters.

This is because when the Scala code is compiled you are effectively calling a chain of methods and your mock needs to set matchers for every possible argument not the ones that your Scala code is actually calling. The simplest solution is to use the any() matcher on any default argument you will not be explicitly setting. Remember that this means the verification must consist entirely of matchers, e.g. eq() and so on.

What to do when you want to verify that a default parameter was called with an explicit value? I think you do it based on the order of the parameters in the Scala declaration but I haven’t done it myself and I’m waiting for that requirement to become a necessary thing for me to know.

Standard
Java, Programming

Python as a post-Java language

I’m a UNIX-based developer and since 2000 I have been working mainly with Java and then JVM languages. When Java 7 slipped I made no real secret of the fact that Java was in a lot of trouble. The post-Oracle world though looks even worse with a lack of clarity of what in the core ecosystem is free, open source and liability free.

Clojure and to a less extent Scala are great steps forward so I don’t feel the burning need for a Java 7/8 whatever. However a moribund or tainted JVM is a major problem and so I’m now thinking about what the post-Java escape route looks like. On the web front it is pretty obvious, Python and Ruby are great languages with great frameworks for developing web-based application. For the server-side heavy lifting it is a lot less clear, people are talking about Google Go but that does feel quite low-level, I’m not sure I’m ready to go back to pointer wrangling even with memory-management. It feels like something you’d build a tool out of not an application. Mono feels like more of the same problems of wrestling with big companies with vested interests, if you are going to do that then why not try and sort out the OpenJDK?

As the title of the post suggests the language I am most inclined towards right now is Python. It is a really concise but clear language that on UNIX systems comes with an amazingly comprehensive set of libraries and which has a virtual environment and dependency management that is on a par with RVM and gem.

The single issue that comes up is performance, what I have been finding that for 80% of the work I am doing performance is okay and I’m producing a fraction of the code I would normally have to create. For that last 20% maybe I am going to have to look at something like Go or (god forbid) go back to C but I would much prefer to see a Clojure or Scala that could run on top of something like LLVM. I also have some hope of smarter people than me making progress on a JIT for Python that might take 20% down to a figure where performance would matter so much to me I wouldn’t mind sweating to make it happen.

Standard
Clojure

Control expressions are functions too

I am kind of simultaneously learning Clojure and Scala, the former through the London Clojure Dojo (come and join us) and the latter through work. It is an interesting experience but I am very glad that Clojure is part of the mix as it helps understand a lot of things that are seem strange in Scala.

In Clojure everything is essentially either a function or a sequence. It is therefore not surprising to see that an if statement works like a function.

(if false 1 2)

Evaluating the above in a Clojure REPL results in the answer 2: if is a function that evaluates its first argument returning the second parameter if the evaluation is true, the third if false.

The same is true of Scala, with the additional wrinkle that if the different clauses evaluate to different types you could be in type hell. Despite its superficial similarity to Java syntax it is in fact quite different and you can compose it just as you would any other function.

1 + (if(true) 2 else 3)

Evaluated in a Scala REPL gives the result 3.

Scala 2.8 seems much better about making everything return a value (and hence act like a function), val and for will both now return Unit, which is not useful but prevents the compiler moaning.

This kind of thing is much easier to appreciate and learn in a pure functional language than in Scala where you never really know whether something is going to operate in a functional or object-orientated way.

Standard
Clojure, Java

Metaprogramming with Clojure

So at one of the recent Clojure dojos we had a situation where we had a number of functions to do with moving that essentially involved passing different keys to a map under different symbols that could be used in the REPL.

Well in Ruby this is the kind of thing you would do by metaprogramming abstracting the shared code into a single function that gets mapped under many names.

During the dojo Tom Crayford provided a map function that seemed to correctly generate the functions we wanted but did so under anonymous names. We couldn’t lick the problem during the dojo and it really drove me a little mad as it seemed we were very close. Some people at the dojo were talking about macros but it seemed that was too strong for the short distance we had to cover to complete the task. It also felt like there was an issue with the language if it couldn’t do this.

After four hours, much consulting of the Halloway book and some fierce googling, a question at Stack Overflow put me on the trail on intern. Adding this to Tom’s function resulted in the right functionality. Here’s a simplified example. However when I ran the code into the REPL the functions failed to appear.

Clojure namespaces are very dynamic and it seems they are very amenable to the kind of manipulation I wanted to do. ns-interns gives you a list of the functions that are currently in a namespace so it was possible to confirm that the functions really had failed to appear. Running the code in the REPL did add them though. So the code was correct.

The final piece of the puzzle was the lazy sequence issue that had been mentioned at the dojo. The map function was being immediately evaluated in the REPL but I had to add a doall to make the sequence unwind when it was invoked during the namespace loading.

The relevant code is part of my Clork fork.

The worst thing in trying to make this work is the old school nature of the Clojure documentation. A lot of the functions tells you literally what the function does but not why you might want to use or more critically give examples of the expected usage. Clojure supports documenting metadata but I think it really needs to be used more effectively.

Standard
Java

Scala and Voldemort: Types matter

So I have been taking a good look at Java object and key-value stores recently. One of the more interesting (because it is very aligned with Java rather than being language-agnostic) is Voldemort. However I got it into my head that it would be a good idea to actually play around a bit using the Scala console. That involved a lot of classpath tweaking, made simpler because the Voldemort distribution comes bundled with its dependencies. However I still rather strangely needed the Voldemort Test jar to be able to access Upper Case View class. I’m not whether I am doing something wrong here or whether the packaging has gone wrong during the build.

With the classpath complete I followed the quickstart instructions and quickly found that actually Scala’s strict type-system was revealing something interesting about the Voldemort implementation. If you translate the Quickstart example to Scala literally then you end up with a Store Client that cannot store anything as it is a Client of type [Nothing, Nothing] and if you supply a value you get an error. I realised that I needed to interact with a lower level version of the API and was surprised to find that the signature of the constructor seemed to need a Resolver instead of defaulting one and that I had to supply a Factory instead of the Config that the Factory consumes.

This is the code I came up with (note that you need to paste this into an interactive Scala REPL rather than putting it in a script). However I’m a nub with Voldemort and I’m not exactly fly with Scala either. Any advice or input is appreciated.

After writing this I was wondering how to try and store arbitrary objects, initially I thought it would be as simple as declaring the Client to be [String, Any] but that failed due to a Class Cast exception in the String Serializer, it looks like the server config comes into play here and says how a value should be serialised and stored. It’s an interesting experience, I have also been looking at Oracle’s BDB but Voldemort puts a much nicer (read Java 1.5) interface on top of it.

Standard