Tuesday, April 13, 2010

The Neo4j REST Server - Part1: Get it going!

WARNING: This post is outdated. The Neo4j Server is now documented here and the manual section on REST is the single source for documentation of the REST API.


As requested and wished by many, finally Neo4j got its own standalone server mode, based on interaction via REST. The code is still very fresh and not thoroughly tested, but I thought I might write up some first documentation on it, based on the Getting Started with REST Wiki page


The first version of the distribution can be downloaded from here: zip, tar.gz. After unpacking, you just go to the unpacked directory and run (on OSX/Linux - see the wiki entry for details on Windows)
$ ./bin/neo4j-rest start
which will start the Neo4j REST server at port 9999 and put the database files under a directory neo4j-rest-db/ (lazily with the first request). Now, let's point our browser (not Internet Explorer since it doesn't send any useful Accept-headers and will get JSON back, this will be fixed later) to http://localhost:9999 and we will see the following:

Things seem to be running! The reason for the HTML interface is the Browser sending Accept: text/html. Now, setting the Accept to application/json will produce
peterneubauer$ curl -H Accept:application/json -H Content-Type:application/json -v http://localhost:9999
* About to connect() to localhost port 9999 (#0)
*   Trying connected
* Connected to localhost ( port 9999 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.19.7 (i386-apple-darwin10.2.0) libcurl/7.19.7 zlib/1.2.3
> Host: localhost:9999
> Accept:application/json
> Content-Type:application/json
* Connection #0 to host localhost left intact 
* Closing connection #0 
  "reference node":"http://localhost:9999/node/0" 

Now, with "200 OK" this is a good starting point. We can see full references to the interesting starting points -the reference node and the index subsystem. Let's check out the reference node:
peterneubauer$ curl -H Accept:application/json -H Content-Type:application/json -v http://localhost:9999/node/0
* About to connect() to localhost port 9999 (#0)
*   Trying connected
* Connected to localhost ( port 9999 (#0)
> GET /node/0 HTTP/1.1
> User-Agent: curl/7.19.7 (i386-apple-darwin10.2.0) libcurl/7.19.7 zlib/1.2.3
> Host: localhost:9999
> Accept:application/json
> Content-Type:application/json
"incoming typed relationships":"http://localhost:9999/node/0/relationships/in/{-list|&|types}",
"incoming relationships":"http://localhost:9999/node/0/relationships/in",
"all relationships":"http://localhost:9999/node/0/relationships/all",
"create relationship":"http://localhost:9999/node/0/relationships",
"all typed relationships":"http://localhost:9999/node/0/relationships/all/{-list|&|types}",
"outgoing typed relationships":"http://localhost:9999/node/0/relationships/out/{-list|&|types}",
"outgoing relationships":"http://localhost:9999/node/0/relationships/out"
Which gives us some info about what the Node 0 can do, how to get its relationships and properties and the syntax of how to construct queries for getting properties, creating relationships etc.

Insert some data

According to RESTful thinking, data creation is handled be POST, updates by PUT. Let's insert a node:
peterneubauer$ curl -X POST -H Accept:application/json -v localhost:9999/node
* About to connect() to localhost port 9999 (#0)
*   Trying connected
* Connected to localhost ( port 9999 (#0)
> POST /node HTTP/1.1
> User-Agent: curl/7.19.7 (i386-apple-darwin10.2.0) libcurl/7.19.7 zlib/1.2.3
> Host: localhost:9999
> Accept:application/json
Resulting in a new node with the URL localhost:9999/node/1 (described by the "self" property in the JSON representation) and no properties set ("data":{}). The Neo4j REST API is really trying to be explicit about possible further destinations, making it self-describing even for new users, and of course abstracting away the server instance in the future. This makes dealing with multiple Neo4j servers easier in the future. We can see the URIs for traversing, listing properties and relationships. The PUT semantics on properties work like for nodes.
We delete the node again with
curl -X DELETE  -v localhost:9999/node/1

and get 204 - No Content back. The Node is gone and will give a 404 - Not Found if we try to GET it again.

The Matrix

Now with properties encoded in JSON we can easily start to create our little Matrix example:

In order to create relationships, we do a POST on the originating Node and post the relationship data along with the request (escaping the whitespaces and others special characters):
curl -X POST -H Accept:application/json -H Content-Type:application/json -d '{"name":"Mr. Andersson"}' -v localhost:9999/node
curl -X POST -H Accept:application/json -H Content-Type:application/json -d '{"name":"Morpheus"}' -v localhost:9999/node
curl -X POST -H Accept:application/json -H Content-Type:application/json -d '{"name":"Trinity"}' -v localhost:9999/node
curl -X POST -H Accept:application/json -H Content-Type:application/json -d '{"name":"Cypher"}' -v localhost:9999/node
curl -X POST -H Accept:application/json -H Content-Type:application/json -d '{"name":"Agent Smith"}' -v localhost:9999/node
curl -X POST -H Accept:application/json -H Content-Type:application/json -d '{"name":"The Architect"}' -v localhost:9999/node

Getting http://localhost:9999/node/1, http://localhost:9999/node/2, http://localhost:9999/node/3 as the new URIs back. Now, we can connect the persons (escaping ruining readability a bit ...):
curl -X POST -H Accept:application/json -H Content-Type:application/json -d '{"to":"http://localhost:9999/node/1","type":"ROOT"}' -v http://localhost:9999/node/0/relationships
curl -X POST -H Accept:application/json -H Content-Type:application/json -d '{"to":"http://localhost:9999/node/2","type":"KNOWS"}' -v http://localhost:9999/node/1/relationships
curl -X POST -H Accept:application/json -H Content-Type:application/json -d '{"to":"http://localhost:9999/node/3","type":"KNOWS"}' -v http://localhost:9999/node/2/relationships
curl -X POST -H Accept:application/json -H Content-Type:application/json -d '{"to":"http://localhost:9999/node/4","type":"KNOWS"}' -v http://localhost:9999/node/2/relationships
curl -X POST -H Accept:application/json -H Content-Type:application/json -d '{"to":"http://localhost:9999/node/5","type":"KNOWS"}' -v http://localhost:9999/node/4/relationships
curl -X POST -H Accept:application/json -H Content-Type:application/json -d '{"to":"http://localhost:9999/node/6","type":"CODED BY"}' -v http://localhost:9999/node/5/relationships
curl -X POST -H Accept:application/json -H Content-Type:application/json -d '{"to":"http://localhost:9999/node/1","type":"LOVES"}' -v http://localhost:9999/node/3/relationships

Now, pointing our browser at http://localhost:9999/node/3/relationships/all will list all relationships of Trinity:

Our first traversal

To start with, the Neo4j default Traverser framework (updated to be more powerful than the current) is supported in REST, and other implementations like Gremlin and Pipes to follow. The documentation on the traversals is in the making here. There are a number of different parameters:
http://localhost:9999/node/3/traverse/node specifies a return type of "node", returning node references. There are other return types such as relationship, position and path returning other interesting info respective. The Traverser description is pluggable and has default values - a full description looks like
"order": "depth first",
"uniqueness": "node path",
"relationships": [
{ "type": "KNOWS", "direction": "out" },
{ "type": "LOVES" }
"prune evaluator": {
"language", "javascript",
"body", "position.node().getProperty('date')>1234567;"
"return filter": {
"language": "builtin",
"name", "all"
"max depth": 2

To note here is the pluggable description of the "return filter" (what to include in the return) and "prune evaluator" (where to stop traversing). Right now only JavaScript is supported for writing these more complicated constructs up, but other languages are coming. Very cool. To finish, let's get all the nodes at depth 1 from Trinity via trivial traversal:
curl -X POST -H Accept:application/json -H Content-Type:application/json -d '{"order":"breadth first"}' -v http://localhost:9999/node/3/traverse/node

Which just returns all nodes of all relationships types at depth one (default) as a JSON Array of node descriptions as above, in this case http://localhost:9999/node/1 and http://localhost:9999/node/2.


Having the Neo4j REST API and with it the Neo4j REST Server coming along is great news for all that want to use a graph database over the network, especially PHP or .NET clients that have no good Java bindings. Already a first client wrapper for .NET by Magnus MÃ¥rtensson from Jayway is underway, and a first PHP client is on Al James' GIThub.
This will even pave the way for higher-level sharding and distribution scenarios and can be used in many other ways. Stay tuned for a deeper explanation of the different traversal possibilities with Neo4j and REST in a next post!


Martin said...

How does that relate to the AGPL licensing? Does this mean Neo4J under AGPL can now be used by closed source code through the REST interface?

Taylor said...

Is there a way to configure the port? Cloud services depend on dynamic ports, so the port is not known until the VM is allocated (then your public port, most likely 80, is bound to the internal dynamic port).

I think just a simple war would have sufficed, but I applaud the intent to make running neo4j simple.


Matte said...

Hi Taylor, the port isn't configurable at the moment and it's just because we haven't got to it yet. It's still considered in alpha state.

However I think it's a minor fix and I'll look into it as soon as I get the time for it!

Anders Nawroth said...

@Taylor: The port is now configurable. Download the latest version (you get that from the links in the post) and see here for instructions: http://wiki.neo4j.org/content/Getting_Started_REST#Configure_port

Lox said...

I wasn't able to get the snapshots working under a 64-bit ubuntu machine. I get a strange error:

Starting Neo4j REST Server...
eval: 1: /usr/local/src/neo4j-rest-standalone-0.8-SNAPSHOT/bin/./wrapper-linux-x86-32: not found

I assure you that the wrapper-linux-x86-32 wrapper is there:

in/wrapper-linux-x86-32: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.0.30, not stripped

Any ideas?

Anders Nawroth said...

@Lox: I have updated the snapshots (same links as before) to include a 64-bit wrapper for Linux. This may be needed on some systems. If there still are problems, try to remove the 32-bit wrapper from the bin directory as well.

Harrolf da Glivor said...

I had some problems to setup Neo4j in Linux in order to code in Python, so I decided to use the new Neo4j REST Server.
However, WADL is too much hard and its syntax in python wadllib hasn't compatibility with current ne4j.py syntax. Thus, I decided to start a implementation of Neo4j Python REST Client: http://github.com/versae/neo4j-rest-client
It's a pre-alpha status, but I think in some days it will be whole functional.


Peter Neubauer said...

Nice Harrolf,
maybe you could announce that Python client lib even on the mailing list? Would be great to see what others think about it ...

Take care,


EmEhRKay said...

Harrof, I just forked it on github. Thanks :)

Harrolf da Glivor said...

EmEhRKay, you're welcome!

Doug Hughes said...

Just wondering, what is the performance of this REST api like? Do the HTTP requests add a lot of latency in your experience? I'm new to Neo4J, but one of the themes I've seen is that it's very fast. That's one of the reasons I'm looking into it. But I want to connect to it from a native Ruby application (not in jRuby) and it seems like REST is the only option. With speed being one of my major concerns, how does the REST API impact things?

Anders Nawroth said...

@Dough Huges: actually CRuby support is coming along in the rjb branch.

Regarding performance over REST, I haven't seen any numbers yet. By exposing a domain-centric REST API instead of using the standard REST component you should be able to reduce the "chattiness" of the communication (this of course depends on the model). The JRuby bindings have support for easily building that kind of stuff.

Pek Chek Kia said...

the query for nodes seem to be slower than mysql query. any chances how to decrease the query timespan?

Anonymous said...

Neo4j guys should take a look Apache SOLR for an easy to use, high performance REST api (the json version of it). It's very easy to bind to any dynamic languages (python/ruby/js), allows for bulk operations (bulk adding data with few requests), and is generally very nicely done.
All this custom HTTP method stuff with lack of bulk operation support largely reduces the usability of the API.

And what's up with all the URL's in the outputs? Who in the world needs a url for a node where a simple node id would be sufficient? same for relationships.

Also, ability to customize output of the path/paths would be great, to include for example all node details, or all relationship details, or just their ids.

Alexander said...

Hi all!
When I run on ubuntu server 11.04 command:
$ ./bin/neo4j-rest start
I have the folowing problem:
Unable to lacate any of folowing binaries:
/home/wonder/neo4j/neo4j-rest/bin/./wrapper-linux-x86-32 (Found but not executable.)
/home/wonder/neo4j/neo4j-rest/bin/./wrapper-linux-x86-64 (Found but not executable.)
Do somebody knows the solution of such a concern?

Thanks for the earlier.

Alexander said...

Sory! I myself found the solution)
It jast needed to execute
sudo chmod 777 *

Mattias said...

Alexander, it looks like you're using an old version of Neo4j. Try a new one from http://neo4j.org/download/