Category Archives: Work

Work

Perverse Zen

Sensible companies plan for poor outcomes, great companies plan not just for bad outcomes but for good ones as well. They look at a range of situations that might occur and create some kind of strategy that will guide the reaction should that situation actually occur.

Most companies however exist in a state of perverted Zen where they are so in the moment they literally cannot see a second into the past or the future. If things are going badly then things are terrible, spending is slashed, projects cancelled and all decisions delayed. If things are going well then things are terrific, projects are approved without any sense of due dilligence, money is spent liberally as there is plenty coming in and the current good times are entirely due to the wisdom and good governance of the current management.

The trouble with this Perverse Zen is that it is entirely reactionary and it makes the whole company vunerable to the whims of whatever mood is currently blowing through the organisation. Good companies I have worked for have always had a range of responses to the future. They also went through the effort of identifying what must be done regardless of what happens to economy, to their customers or indeed to the structure of their business.

These priorities then inform what the response to a given situation will be. When times get bad cutting effort to zero is often counter-productive, you want to match effort to your capability to spend. Similarly when times are good it helps to already have a plan as to how to spend sensibly; bringing about the largest and most immediate benefit to the organisation.

Standard
Web Applications, Work

The death of MVC

The MVC pattern is so embedded into the concept of modern web development I feel quite the heretic for declaring it over. Yet more and more I think we are moving away from it as a pattern. Views have been boiled down to a special case of templating and now Controllers are the next under the microscope. What does a Controller do? Well it marshalls the model and exposes it to the view.

However with the relentless march of REST how much controlling is the Controller really doing? The HTTP request tells you the format of the data, identifies the resource it is interested in. How much need is there for a controller for each request type? Surely the Uber Controller that responds to the HTTP request is all that’s required.

I have also been using Groovlets recently and when using them I feel like “why not mix your model lookup with the ‘view'”. In my Groovlets I essentially lookup the data for the view either directly via Groovy SQL or via the Service layer that is injected out of Guice. The view is then created using Markup Builder.

Since my scripts end up at around 50 lines of code I think that any benefit I might have in separating things is outweighed by the fact that the entire interaction is in one place and can be found, read and changed very easily.

MVC saved us from really painful web architectures but as we grow more sophisticated in the way we handle HTTP Requests and the more we understand the implications of HTTP and the less ceremonial our languages becomes the less benefit we get from it.

Standard
Software, Work

Offshoring means your job too

I quite often come across a strange attitude amongst people in favour of offshoring development work. It reached a kind of apogee in a recent quote that “The thinking will remain in the UK but the work will be done abroad where it is cheaper.”

Don’t kid yourself! If you outsource the work then you also outsource the thinking. If you are an architect, or a designer and your implementation team is offshored then your work has also been offshored. Without the vital feedback from your implementers your high-level input is very quickly going to go stale. The understanding of how to do a job lies with those who do it, not those who raise the purchase orders.

Don’t flatter yourself! For some reason a lot of people in the UK seem to think that the IT staff in places like China, Eastern Europe and India are incapable. I know from experience that there are more smart people in all of these countries than there are in the UK. It’s a simple case of numbers. Don’t confuse offshoring to a lowest cost bidder with being incompetent. You pay so little for your code that it sends a message: we don’t care about our software. If you don’t care why is the wage slave that is eventually contracted to produce it?

So if you are involved in “higher food chain” work, don’t get caught up in the hype. The thinking will always end up following the doing, and then, ultimately, the purchasing does as well.

Standard
Work

The Dark Side of Employee Empowerment

It is a great thing to empower your employees to make a change within your business. It is another to actually empower them to make change rather than just saying it.

However empowering your employees does not give you a get out of jail free card. To often management respond to criticism by shrugging and saying that if the staff don’t like something then they should change it.

If an employee doesn’t like the strategic direction of the firm they work for they really cannot just go out and change it. The most they can do is lobby and harp at management to try and change direction. If they see little immediate reaction to their badgering then they are quickly demotivated to campaign for changes.

Empowerment does not mean that management should not actively seek feedback and act on it, particularly if change is easy for them to effect. They should also be ready to shoulder the burden that if you empowered your employees and you still have problems then they are still your problems. Saying that employees should have fixed them doesn’t help your staff or yourself.

Standard
Software, Work

What’s the difference between Simple and Stupid in interview code?

Kent Beck (I believe) said: “Do the simplest thing that works, not the stupidest”. What does that mean in the context of showcase code (i.e. code you have written as part of some kind of application process)?

Well firstly think of what the point of showcase code is. What is the reviewer looking for as part of the application process? They are looking for evidence of a train of thought or method of problem solving that chimes with the kind of thought processes that they think are effective. Ideally these would be the “best” processes but you have to accept that a lot of processes are about finding people who agree with what an organisation thinks is best practice rather than respecting originality and clever solutions. You should never try to tailor your showcase code to an organisation because it is too hard to double guess what someone is looking for. You can only express yourself as clearly as you can and hope that it fits well with what is being sought.

A good piece of showcase code will appropriately abstract the problem being set, illustrating the way the candidate finds solutions to problems. It will neither be too elaborate, abstracting too much of the problem, nor too literal, coding “between the lines” of the problem being set.

Let’s take an example. If the problem is to model a set of automobile configurations in an object then the problem set may explain that a car is sold with seat configurations of 2, 4 or 6. In my view a stupid solution would be to have a list of just 2, 4, 6. Clearly the reviewer is going to want some flexibility here, the solution should be able to deal with a new configuration of 5.

An obvious solution is to just model the number of seats as an integer value. You can encapsulate the data by providing an API that allows you query how many seats a configuration has and provide a way of setting that value in an immutable way (once a configuration is set then the number of seats it has is unlikely to change in the lifetime of the object, that’s simplicity not stupidity). This solution just exposes that a configuration has a number of seats which is an integer, which is logical. It is an entirely sensible solution that keeps it simple.

How might a nervous candidate overcomplicate this situation? Well the number of seats really is immutable unless the problem set says otherwise, you don’t need to supply a mutator. Similarly the number of seats is never going to be a Real number.

What about modelling the number of seats as a list of Seat objects? Well yes that works because the encapsulation should hide the implementation and the number of seats is now just the number of Seats in the Seat list. Domain-driven design fans might even say this is a better solution that an integer because it is a closer description of the domain. However I think that simplicity would tend to demand that until we have something else to model about a Seat (say, whether it has a seatbelt) then it is hard to justify having a class whose only purpose is to be counted in a list. For all the difference it makes I could put Turtle objects into the Seat list. In terms of showcase code using this technique actually gives the reviewer a bit of a headache because instead of just ticking the box and moving on the reviewer now has to ask whether the new level of abstraction introduced is valid or not, is it a good idea to have a class with no functionality or data? What if actually the seat configurations are better represented by enumerated constants? The effort of creating the Seat class is just wasted and has to be replaced.

This solution at least stays within the bounds of an existing paradigm. If a candidate starts to abstract wildly then reviewer is going to give up in frustration. If the candidate starts abstracting the model to the point where it could model a motorcycle or truck just as well as car then they have just failed the process. Unless the problem says something about having a motorcycle then you don’t want to see Vehicle With An Engine type classes. After all if you start to generalise then where do you stop? Sure you can model a motorcycle and a truck but why not a plane? Or a rocket?

Standard
Software, Work

Don’t hate the RDBMS; hate the implementation

I read through this post with the usual sinking feeling of despair. How can people get so muddled in their thinking? I am not sure I can even bear to go through the arguments again.

More programmers treat the database as a dumb store than there are situations where such treatment is appropriate. No-one is saying that Twitter data is deep, relational and worthy of data mining. However not all data is like Twitter micro blog posts.

The comments on the post were for the most part very good and say a lot of what I would have said. However looking at CouchDB documentation I noticed that the authors make far less dramatic claims for their product that the blog post does. A buggy alpha release of a hashtable datastore is not going to bring the enterprise RDBMS to its knees.

I actually set up and ran Couch DB but I will save my thoughts for another day, it’s an interesting application. What I actually want to talk about is how we can get more sophisticated with our datastores. It is becoming apparent to me that ORM technologies are really making a dreadful hash of data. The object representation is getting shafted because inheritance is never properly translated into a relational schema. The relational data is getting screwed by the fact the rules for object attributes is at odds with the normal forms.

The outcome is that you end up with the bastard hybrid worst of all worlds solution. What’s the answer?

Well the first thing is to admit that application programmers think the database is a big dumb datastore and will never stop thinking that. The second is that relational data is not the one true way to represent all data. They are the best tool we have at the moment for representing rich data sets that contain a lot of relational aspects. Customer orders in a supply system is the classic example. From a data mining point of view you are going to be dead on your feet if you do not have customers and their orders in a relational data store. You cannot operate if you cannot say who is buying how much of what.

If you let developers reimplement a data mining solution in their application for anything other than your very edge and niche interests then you are going to be wasting a lot of time and money for no good reason. You simply want a relational datastore, a metadata overlay to reinterpret the normalised data in terms of domain models and a standard piece of charting and reporting software.

However the application programmers have a point. The system that takes the order should not really have to decompose an order taken at a store level into its component parts. What the front end needs to do is take and confirm the order as quickly as possible. From this point of view the database is just a dumb datastore. Or rather what we need is a simple datastore that can do what is needed now and defer and delegate the processing into a richer data set in the future. From this point of view the application may store the data in something as transient as a message queue (although realistically we are talking about something like an object cache so the customer can view and adjust their order).

Having data distributed in different forms across different systems creates something of headache as it is hard to get an overall picture of what is happening on the system at any given moment. However creating a single datastore (implemented by an enterprise RDBMS) as a single point of reference is something of an anti-pattern. It is making one thing easier, the big picture. However to provide this data is being bashed by layering technologies into all kinds inappropriate shapes and various groups within the IT department are frequently in bitter conflict.

There needs to be a step back and IT people need to accept the complexity and start talking about the whole system comprising of many components. All of which need to be synced and queried if you want the total information picture. Instead of wasting effort in fitting however many square pegs into round holes we need to be thinking about how we use the best persistence solution for a given solution and how we report and coordinate these many systems.

It is the only way we can move forward.

Standard
Groovy, Programming, Scripting, Work

Using Groovy to create XML Schemas

I have previously found XML Schema to completely invaluable in defining interface points between systems. Normally file interfaces between systems are done in formats that are deceptively simple: CSV, structured text files. However in nearly all cases the initial simplicity tends to lead to a lot of problems. There is the issue of how to escape characters within your fields, particularly the field separator. The free text field is often used as exactly that, free text. Something is supposed to be a number field… until the letter X appears in it. Historical CSV is the worst as often the exact meaning and origin of each of the fields is undocumented and the meaning lost. I have even come across CSV generators that map meaningless constants to the output just to keep the number of fields the same. The receiving systems ignore those same fields or sometimes even hinge workflow off a value that will never vary in practice. The whole thing ends up being a nightmare.

Introducing an XML Schema reduces that nightmare but does bring in a lot more complexity. Being able to specify the type and order of the fields comes at a price. Previously when I have wanted to develop a new schema I have simply used the Xerces tools at the command line and an XML editor to generate both the Schema and a sample datafile. It works but it is quite laborious. Speeding this up would be great as often the point of capturing the complexity in the data transfer is so that the business or the architects can see the complexity of the integration and decide that they really want to do it before a lot of code gets written to integrate the systems.

Looking through the Groovy website I came across this example of how to validate an XML document and an idea is sparked. The multi-line indicator is a neat feature (borrowed from Python I think) and is (to my mind) a more elegant solution than the Ruby/Perl document syntax. It would allow me to define my schema, my sample document and my validation code in the same file. During iterations I would be more productive and when the interface is captured I just publish the final schema.

So I’ve knocked together a simple PoC and it seems to work pretty soundly. The easiest way to work with it is from the sample document to the Schema but TDD approach is to define the Schema and work back from the validation errors. The latter approach tends to avoid the situation where you’re validating your test document rather than your document template.

Standard
Java, Ruby, Web Applications, Work

Working with Rails

I have recently struggled to try and get a prototype web application done with Ruby on Rails. It has been a really great experience and like all such experiences consists of both good and bad moments. Things that are particularly striking coming from a Java Web environment:

  • Integrated console
  • No recompile/build/deploy cycle
  • No mix and match components
  • No real programming language skill required.

It is also remarkable how quickly people revert to the level of thought, planning and execution that you might use for a shell script. Even I have found myself getting caught up in “playing” with the app interactively rather than being focussed on creating behaviour and functionality in a structured way.

Elaborating on these points; in Java web development the first issue is usually building up your component stack. Some people just choose Spring  but those people are idjits! If you are working purely in a web tier without any need for any backend interaction then it is probably a good bet but generally you want to put some thought into how you are going to assemble your various stacks. You tend to have to choose your MVC web framework, your persistence framework and your service framework (if appropriate). You often have to give some thought to your caching and messaging frameworks if they are relevant.

With Rails you just use the appropriate Active Module or Rails built-in features. If there is something that is going to ease some pain for you then it is going be operating as a plugin. That’s it, no framework holy war. It also means that there are a lot of applications that are just not going to be suitable for a Rails application as all those Java components have various strengths for different environments. I certainly wouldn’t want to tackle a legacy database with ActiveRecord for example. Not that it couldn’t be done but I wouldn’t want to do it.

The zero-turnaround is impressive after the build-deploy-check cycle. It was a wow factor last year when I saw the Phobos framework being demonstrated by Sun and its not lost any of its shine. By separating its deployment environments so ruthlessly Rails is able to deliver a really positive developer experience.

The interactive console takes a bit of getting used to in terms of faking browser requests but once you get used to doing so it is another tool that you wonder how you’ve managed without for so long. It’s much more intuitive to use than setting up a remote Java debugging instance (although admittedly you do similar tasks in both). It allows you to scratch those itchy “why?” questions.

And finally that lack of programming skill? A bit controversial perhaps? Well I feel that in recent weeks what I have been learning is how to manipulate Rails. The fact that it is implemented on Ruby may allow a lot of features to be implemented in the way they that they are but you are very rarely called upon to show Ruby madskillz. Instead the majority of the time you are simply plugging little customisations into the Rails framework. It is called Rails for a reason after all, when you are on them you are amazingly productive but if you can’t package your problem into a Rails solution then you are out of luck, you’re going to have to develop your own solution and that is going to be hard work. I have been trying to develop my Ruby skills (that is a whole other story) but the truth is you can bang out something that is acceptable with very little Ruby knowledge. You can go a lot of the way purely by mastering Ruby’s hash syntax.

So have I been converted? Well for all its problems I do have a hankering to get back to my great big Java applications with their holy wars and heavyweight processing. After all being on the Rails is fine for getting things done quickly but it can feel claustrophobic. I am also really glad that after getting distracted by the whole JSF controversy the challenge Rails presents to the status quo of web development means that a lot of the Java frameworks are starting to respond to the real problems faced in web development and ensuring that the easy stuff should be easy.

Standard
Work

Entrepreneurs versus Businessmen

A discussion over SLAs recently helped me define one difference between businessmen and entrepreneurs. Entrepreneurs generally have a huge appetite for risk, they know that things might go wrong and that they might go wrong catastrophically but they go ahead and do something anyway. A businessman on the other hand waits until something is so safe that anyone could do it.

For the entrepreneur doing and doing it first is everything. For the businessman it is about doing it right and hopefully doing it better than everyone else.

Standard