A weird edge case came up today around a buildpack that turned out to be unneeded for my application. How do you change the Buildpack once you’ve associated it with your application? It turns out that a Buildpack is essentially just a piece of config, so you can see it and change it by using the heroku config command and the config variable name BUILDPACK_URL.
In my case I just needed to remove the configured Buildpack with a simple heroku config unset command:
heroku config:unset BUILDPACK_URL
Okay this is quite specialised by the four or five of you who will have the same problem I wanted to save you some time and suffering.
So Neo Technologies have been incubating a plugin of their excellent graph database on Heroku for a while. So far the plugin was only available in beta but now anyone can have it. This is excellent news and I would recommend it as a way of getting starting with graph-based web programming. However if you were in the beta program then you now need to migrate from the beta plugin to the new one.
The instructions that went out to the beta program implied that this was simply a case of dumping a backup zip, switching out plugins and then uploading your zip. Well the good news is that exporting and importing the zips works exactly as advertised but the bad news is that the two plugins are quite different in terms of the environment they expose. The beta plugin had an extensive list of variables that exposed the various parts of your hosted environment. The new one just exposes the variable NEO4J_URL which is a url to the server and contains an embedded username and password.
Now the new variable does actually encode all the information that the original manifest did but in a very limited way and your library is going to have to work quite hard to correctly construct the base urls and requests required to access the REST API. I’m not sure which libraries do do it (I presume the Java ones) but neither of the Python ones do.
I’m going to describe what you need to do for the neo4j-rest-client which is the one I use in my apps but it will probably be similar for py2neo which is what you might want to use if you want to use a lot of Cypher.
So the simplest way to explain the solution is code.
import urlparse
from neo4jrestclient.client import GraphDatabase
def extract_credentials(url):
parsed_url = urlparse.urlparse(url)
if parsed_url.username and parsed_url.password:
return (parsed_url.username, parsed_url.password)
return None
GRAPH_URL = os.environ.get('NEO4J_URL', "http://localhost:7474") + "/db/data/"
credentials = extract_credentials(GRAPH_URL)
if credentials:
db = GraphDatabase(GRAPH_URL, username = credentials[0], password = credentials[1])
else:
db = GraphDatabase(GRAPH_URL)
So the neo4j-rest-client library supports username and password credentials but doesn’t parse them out of the url itself. Fortunately urlparse makes this pretty trivial. The conditional pieces of the code deal with the situation where we are running locally, essentially if we can’t see the Heroku environment variables we want to fallback to the local case (most Heroku stuff works this way).
A more frustrating issue is the difference between the url of the server and the root resource for the REST API. Naturally these are not the same but few libraries handle been given the wrong url that gracefully. Since the host URL does return successfully you usually get some failure about parsing or unpacking the root document. Submitting a patch to detect where the url ends in db/data or not would seem to be the logical solution.
So this code should boot a successful wrapper around the REST interface and your app should work again.
Except, that there seems to be another issue in the registering and deregistering of the plugin manifests. What I have observed is that heroku config lists the beta environment variables and not the new values. So even if you do this the library still gets 404 errors on the root document (because it is looking for the Neo4J environment that has been deallocated).
So the best way to migrate your app in my view is:
This seems easier and less hassle than migrating in place. Once the beta plugin is turned off you should be able to delete the old app.
This process has allowed me to migrate my two demo apps successfully (pending bug reports): Crumbly Castle and Flow Demo.
Last week I gave a talk on how I have been creating web applications that very lightly wrap an underlying graph to provide not just content for a page but also the workflow and state of the user’s current interaction with the application.
As part of the talk I have created two demo apps that are available on Heroku. Crumbly Castle is inspired by Dark/Demon Souls and allows you to explore a castle that is populated by the ghosts of everyone who has ever played it. The other offers a questionnaire system that generates characters in the style of the Elder Scroll or Fallout games. The code for the applications is on Github so you can fork it and deploy it for yourself. Both use the hosted Neo4J addon for Heroku which provides hassle-free hosting but is currently only available to beta program members.
You can obviously use both on your local machine.
Both of the demos are metaphors for more serious kinds of enterprise applications but I think it is often easier to produce prototypes or demos that are based on immediately engaging concepts. It certainly helps to have something that the audience can play with during the talk!
So briefly I just wanted to summarise the points I try to make during the talk and explain why you might want to look at using a graph as your web application store. So my major point is that web application development is usually page-centric, when you hit a page the controller tends to examine the whole state of the application to find out why you came to the page. Are you logged in? Were you trying to look at something? Is there a session associated with you?
I posit that we should instead be looking at the journeys between the pages as being the interesting things. Given where you are in the journey graph where can you go next? Essentially I am taking the same logic as a state machine or rule engine uses and instead expressing it as a relationships in a graph.
The most common trick the applications use is to assign a fixed url to a user session that identifies a node in the graph. Then with each transition I change the relationships the node has to other data based on the user’s actions and then simply send a redirect back to the fixed url which will then render a different result based on the current state of the node.
This means that the web application becomes very simple to write and the controller simply has to select the template and the related nodes that are needed to generate links and actions.
I think it is a really interesting approach that is a really natural fit for simplifying a lot of session-state heavy apps.
Heroku allows you to use CouchDB via the Cloudant cloud service which is great but compared to the documentation for the relational stores it is not clear how you are meant to deal with backups and importing of data. I also couldn’t find a way to use Futon on the Heroku instance (which comes from the Heroku account, you can’t use your own Cloudant account with the plugin) or share the database instance with my personal Cloudant account.
This post from Cloudant helps a lot, essentially you can get your Heroku instance URL and then the cool thing about Couch’s painless replication is that once you have a Couch URL you can replicate that database to a local instance or even back into Cloudant.
heroku config --long
curl CLOUDANT_URL/_replicate -H 'Content-Type: application/json' -d '{"source" : "CLOUDANT_URL", "target" : "TARGET_URL"}'
You can edit the database locally and then replicate back to the Heroku instance by just swapping the URLs in the Curl above.
That seems to pretty much be it. I’ve replicated my data out of Cloudant and then back into it, which feels bizarre but it’s all symmetrical with Couch and it’s a handy cloud-based backup mechanism.
At the Geek Night last night there was a lot of questions about how Heroku compared with Google App Engine (and I guess implicit in that how it compares to GAE/J with the JRuby gem). Since it came up several times I thought it would be interesting to capture some of the answers.
Firstly there are the similarities, both deal with the application as the base unit of cloud scaling. Heroku offers several services that are similar to GAE, e.g. mailing, scheduled and background jobs. Some aspects like Memcache are in beta with Heroku but fundamentally both services seem to concur.
They are both free to start developing and then the pricing for the system varies a bit between the two, GAE might be cheaper but also has a more complex set of variables that go into the pricing, Heroku prices on: data size, web processes required and then options, so it’s easier to project your costs.
Differences then: well a big one is that Heroku uses a standard SQL database and also has excellent import/export facilities. GAE uses Google BigTable which is great for scaling and availability but data export and interactions are very limited. Heroku has an amazing deployment mechanism that involves simply pushing to a remote branch. GAE is slightly more involved but it is still relatively easy compared with deploying your own applications.
GAE provides access to a variety of Google services that are handy such as Google Account integration and image manipulation. GAE also has XMPP integration which is pretty unique if you have the requirement. However these do tie you to the platform and if you wanted to migrate you’d have to avoid using the Google webapp framework and the API. If flexibility is an issue you are probably going to prefer Heroku’s use of standard Ruby libraries and frameworks.
Heroku has few restrictions on what can and can’t be done in a web process, you also have quite powerful access to the user space your application runs in. GAE and particularly GAE/J has a lot of restrictions which can be quite painful particularly as there can be a discrepency between development and deployment environments.
However running a JVM in the cloud provides a lot of flexibility in terms of languages that can be used and deployed. Heroku is focussing on Ruby for now.
From my point of view, choose Heroku if you need: uncomplicated Rails/Rack application deployment, a conventional datastore that you want to have good access to, a simple pricing model. Choose GAE if you want to have out of the box access to Google’s services and a complete scaling solution.
Heroku is an amazing cloud computing service that I think is incredibly important but which really isn’t getting the visibility it should be at the moment.
Why does Heroku matter?
Firstly it gets deployment right. What is deployment in Heroku? It is pushing a version of your app to the cloud. That’s it. The whole transformation into a deployable application slug is handled by the Heroku, pushing it out to your dynamos is handled by Heroku. Everything is handled by Heroku except writing your application.
Secondly, it is the first cloud/scalable computing targeted directly at Ruby that has the right pricing structure to encourage experimentation and innovation. It has the low barrier to entry that Google App Engine has but requires no special code and imposes no special restrictions.
Thirdly it does the right thing with your data. It respects your right to access, mine and manipulate your data and provides a mechanism for doing this easily.
For me Heroku has set a new benchmark for Cloud Computing and build and deployment in the enterprise as well.
Echo One 