Thursday, May 26, 2011

Announcing Neo4j 1.4 M03 “Kiruna Stol”




Today marks our third milestone in the Neo4j 1.4 releases. We’ve spent the time since our last release listening to the community and adding to our APIs to help make working with the database even easier and more productive. Under the covers we’ve also built in some performance enhancements that we think you’ll appreciate. And our eye-candy, which you know as Webadmin, has also been extended and tweaked.

Loop-the-loop

As the community has been working through new use-cases, we’ve heard that creating a relationship from a node back to that same node (so called self relationships) has become a desirable feature. As of this release, self relationships are supported by the Neo4j kernel. We’ve decided to implement the feature in the least disruptive way possible, using the existing Node.createRelationshipTo(...) method, which now allows you to use the same destination node as the originating node. Be aware that the kernel won’t complain about loops, so make sure to test that you don’t accidentally create them and cause your traversals to misbehave!

Webadmin indexes

Our Web wizards have been hard at work adding more functionality (and more bling) to the Webadmin tool. You can now interact with your indexes though Webadmin, which allows you to create and delete node and relationship indexes from the Index Manager tab, or you can query them from the Data Browser tab, which now accepts an extended query syntax.

Faster, smaller, smarter caching

Down in the kernel, we’ve implemented a novel way of caching relationships based on their direction. For those nodes which have many (millions) of relationships, iterating over those relationships will be much faster than before. What’s particularly nice about this upgrade is that you don’t need to change your code, nor are there any changes to the Neo4j API. That’s full of win!

Highly available, less highly confusing

In Neo4j HA we’ve built a new algorithm that favors a stable master database over time in a cluster. This prevents the hiccup where ZooKeeper could cause unnecessary (inconvenient and annoying) master switches.

And finally...

All this goodness is now available from our download site. Grab it, play with it and give us your feedback on the mailing list. Don’t forget to be safe - this is a milestone release so always back up your data! Enjoy!

Ian Robinson
Director of Customer Success, Neo Technology

(For those of you who haven't yet rushed off to grab the release, and are still reading, a belated introduction. I'm Ian Robinson, and as you can see, I have the delightful title of Director of Customer Success. That means I'm here to help make you wildly successful in everything Neo4j and graph related. I'm based in London, but expect to see me pop up wherever Neo4j is being used to address your complex data issues. Over the next few weeks, we're going to make it even easier for you to give us your feedback, make suggestions, and raise issues. In the meantime, you can contact me at ian.robinson@neotechnology.com, or through the mailing lists.)

Friday, May 13, 2011

Neo4j 1.4.M02 - Index-ability

NOTICE: This version introduces an upgrade in the index store format that is not backwards compatible. Be sure to backup all data before trying out and make sure you do not test this release on production data before extensive readiness testing.


Another milestone goes by as we pick up speed on the way to the “Kiruna Stol” release. This time we have a host of performance improvements coupled with additions to the REST API.

Win-dex!

One of the most striking features of this milestone release is the new index component based on Lucene 3.1. Besides the cleaner API, this upgrade brings impressive improvements in index lookup speed, with index write operations enjoying measurable speedup as well. So, depending on your use case, you might benefit from indexing operations that are twice as fast!

To take advantage of the new index components, you’ll need to upgrade your index store, since Lucene itself has upgraded its store format in a way that is not backwards compatible. While Neo4j will happily upgrade your index store, you should be aware that upgraded stores are not backwards compatible with previous Neo4j versions. So, as always, backup before upgrading and do not try this milestone version in production without extensive testing.

Not REST-ing on our Laurels

Neo4j’s REST interface hasn’t been at feature parity with the Java APIs in the indexing area. With this milestone release, we’ve put an end to that disparity and now all index operations are supported both through the Java APIs and through the REST interface.

For example, the REST interface now supports arbitrary Lucene queries, deletion of indexes and index lookups on default keys. So, deleting the node index named myNodeIndex can be done via HTTP as follows:

curl -X DELETE http://localhost:7474/db/data/index/node/myNodeIndex

Or, to do a generic query on node index named my_nodes for wildcard key the_* and range [1 TO 100]:


curl -H Accept:application/json http://localhost:7474/db/data/index/node/my_nodes?query=the_key:the_%2A%20AND%20the_other_key%3A%5B1%20TO%20100%5D

For complete information on the new REST operations against the index you can always look at the documentation at http://components.neo4j.org/neo4j-server/milestone/rest.html

Better, Faster, Smaller

An important technical change introduced with this release is reverting the default implementation of the Write Ahead Logs from using memory mapped files to a buffer backed equivalent. This should provide you with significant speedups, especially in code that works with small transactions, where our measurements show an increase in throughput of up to 20 times! This scenario is most commonly encountered when working with the server, so users of the REST API should rejoice. We are anxious to hear how this improvement affects your experience with Neo4j.

So, go ahead and download the latest “Kiruna Stol” milestone and let us know what you think!