Thursday, December 30, 2010

Neo4j 1.2 Final - Prêt-à-Porter



With the Neo4j components, we supplied the fabric for stitching together graph-based applications. Now, with Neo4j 1.2 we are proud to offer a ready-to-wear graph server, for dressing up any application in graph style. This is the start of a new trend, as we incorporate more of the components into services which are easily accessible from any language, on any platform.

NOSQL Graph Database Super Model

A classic Neo4j graph model.

Neo4j is one of the fashionable Not-Only-SQL alternatives to traditional storage, representing the graph database category. A graph database is a natural fit for any data, optimized for accessing related records rather than aggregating data. The "records" are nodes or relationships, because both can store properties.

From the filesystem up, Neo4j is designed for efficiently storing and querying graphs of data. Whether using the new server or embedding the components in your application, you get the same robust graph database, ensuring durability, consistency, and ease of access. Let's listen to designer Emil Eifrem on the subject ...

A New You in 1.2

It's fun to say: Neo4j now runs as a server! Running as a server hides some of our Java underwear, presenting a respectable RESTful interface to applications written in any language, running on any platform. Ruby rock stars, Pythonistas, PHP script heads, C# sharpeners (hm, that doesn't quite work), you are all welcome here.

The server's REST API is well documented and discoverable, exposing the essentials of graph services: nodes and relationships, indexing, traversals, and a few algorithms. If it doesn't quite suit your tastes, you can use the Plugin API to add low-level operations which are then available through REST. Perhaps a new traversal algorithm, or a set of nodes representing an entire application entity. Whatever you'd like.

Of course, a real server should run as a system service. With appropriate privileges you can easily install Neo4j as a Windows service, or as a Unix daemon, participating in the proper startup and shutdown process of the operating system.

And, it has a lovely web-based administration interface ...

When you're ready for massive exposure, you'll want to consider ...

Being Highly Available

For demanding applications, the High Availability (HA) feature of Neo4j provides an easy way to set up a cluster of graph databases. You get read scalability that tolerates faults in any participating machine. Writes are allowed to any machine, though synchronized with a slight delay across all of them.

While useful today, Neo4j High Availability is a sneak preview with a few limitations. It does provide scalability for read load, but write operations are slightly slower. Adding new machines to a cluster still requires some manual work, and very large transactions cannot be transmitted across machines. These limitations will be addressed in the next version of Neo4j.

Custom Tailored

But 1.2 news isn't all about the server. Embedded developers will enjoy some new components, and also updates to existing ones.

1. Kernel

The kernel has a much smaller memory footprint. Read performance has improved significantly. And, there is a new weak reference cache which is perfect for high load, low latency workloads.

A new indexing API is integrated with the GraphDatabaseService, better aligning the lifecycle of the service and making the indexes a natural complement of the graph.

Finally, the kernel now has service extensions that load automatically, or can be injected in a running instance. Examples include the Neo4j shell server and the management component.

2. Graph Database Monitoring and Management

Monitoring and management has been moved into its own component that will be loaded automatically as a kernel extension. This component lets the graph database instance participate in your JMX management infrastructure.

3. UDC

The Usage Data Collection component provides built-in feedback for guiding the development of Neo4j. More about that can be read in our previous blog post on the subject.

For more details about these and all the components, check out the components website.

Strut Your Stuff

OK, enough about us. Let's talk about you. Show us what you can do.

As always, we're grateful to our great community of contributors, both of code and comment. Thanks everyone!

Cheers to the New Year!

Sunday, December 12, 2010

Neo4j 1.2 M06 is out - Better REST Indexing and Server Plugins!

Peter Neubauer

Hi all, the Neo4j community is proud to release another milestone into the wild - Neo4j 1.2 M06. This time the main effort has been spent in two areas - a more complete REST API, and the first version of a plugin API for the Neo4j Server.

There has been a lot of feedback on the REST API and the server, thanks to all contributors, mailing list discussions, and real customers!

 

Integrated REST Index API

The new Index API that is integrated with the GraphDatabaseService is now exposed through REST, meaning that you can access the same indexes created through embedded Java. Rather than just a single index for nodes and another for relationships, it is now possible to have multiple named indexes of each type. Adding a node to an index automatically creates that index if it doesn't already exist.

For instance, to add a (pre-existing) node to a node index called "persons" you would POST:

curl -HContent-Type:application/json -X POST -d '"http://localhost:7474/db/data/node/123"' http://localhost:7474/db/data/index/node/persons/name/peter

Then, to look up all nodes that have a name property of "peter" in the "persons" index, simply GET:

curl -H Accept:application/json http://localhost:7474/db/data/index/node/persons/name/peter

For more thorough documentation of the REST API, see the API docs. They are even packaged with the download.

 

Server plugins - Hypermedia at work

Plugins provide an easy way to extend the Neo4j REST API with new functionality, without the need to invent your own API. You simply populate an Iterable of Node, Relationship or Path, specify parameters, point of extension and logic, and voila, you are done! Look at this:

 

 

@Description( "An extension to the Neo4j Server for getting all nodes or relationships" )
public class GetAll extends ServerPlugin
{
@Name( "get_all_nodes" )
@Description( "Get all nodes from the Neo4j graph database" )
@PluginTarget( GraphDatabaseService.class )
public Iterable<Node> getAllNodes( @Source GraphDatabaseService graphDb )
{
return graphDb.getAllNodes();
}

 

@Description( "Get all relationships from the Neo4j graph database" )
@PluginTarget( GraphDatabaseService.class )
public Iterable<Relationship> getAllRelationships( @Source GraphDatabaseService graphDb )
{
return new NestingIterable<Relationship, Node>( graphDb.getAllNodes() )
{
@Override
protected Iterator<Relationship> createNestedIterator( Node item )
{
return item.getRelationships( Direction.OUTGOING ).iterator();
}
};
}
}

 

This will make your extension visible in the database representation (@PluginTarget) whenever it is served from the Neo4j Server (this could also be Node or Relationship), letting clients automatically discover it from e.g. at http://localhost:7474/db/data:

 

curl -v http://localhost:7474/db/data/

 

{
"extensions-info" : "http://localhost:7474/db/data/ext",
"node" : "http://localhost:7474/db/data/node",
"node-index" : "http://localhost:7474/db/data/index/node",
"relationship-index" : "http://localhost:7474/db/data/index/relationship",
"reference_node" : "http://localhost:7474/db/data/node/0",
"extensions" : {
"GetAll" : {
"get_all_nodes" : "http://localhost:7474/db/data/ext/GetAll/graphdb/get_all_nodes",
"getAllRelationships" : "http://localhost:7474/db/data/ext/GetAll/graphdb/getAllRelationships"
}
}

 

Requesting a GET on one of the two extension endpoints...

curl http://localhost:7474/db/data/ext/GetAll/graphdb/get_all_nodes

 

...gives back the meta information about the service:

{
"extends" : "graphdb",
"description" : "Get all nodes from the Neo4j graph database",
"name" : "get_all_nodes",
"parameters" : [ ]
}

To use it, just POST to this URL, with parameters as specified in the description (none in this case).

This way, plugins contribute hypermedia and still maintain the notion of a tight REST API which talks Nodes, Relationships and Paths. Some very real plugins we are thinking of in the near future (and that are less than 100 LoC):

 

  • global functions with parameters, like getting some named nodes by id.
  • custom traversal implementations
  • server-side scripting wrappers that can execute scripts (JavaScript, JRuby, Gremlin etc) installed in some server directory
  • plugins that let you execute a scripted payload from the request

We have put a copld of example plugins into the download under /eamples/java, see http://components.neo4j.org/neo4j-examples/1.2-SNAPSHOT/ for online docs on them.

 

All in all, we think that we are getting pretty close to Neo4j 1.2 now, so stay tuned for the big push!

Your friendly Neo4j community.