Neo4j 1.3 “Abisko Lampa” M04 – Size really does matter


Note: The store format changed in this milestone to allow for large databases. This means that you manually need to run an upgrade before your old database works with Neo4j 1.3.M04 and later. Please see me after class for more info. Wait, no: I mean see the latest documentation.
Champion Andrés Taylor (in the middle)

Today we passed the fourth milestone on the tracks towards Neo4j 1.3 Abisko Lampa. By far the greatest change in this release is how much data Neo4j can store for you (hint: approximately 128 million truckloads). We have also improved our quality assurance on Windows, and extended our REST API to cover more of what Neo4j embedded can do.

Big Store Opening

A database can now contain 32 billion nodes, 32 billion relationships and 64 billion properties. Before this, you had to make do with a puny 4 billion nodes, 4 billion relationships and 4 billion properties. Finally, every single person on Earth can have their own personal node! And did we mention this is happening without adding even one byte to the size of your database?

This is a big change in the store format, though. It is very important to upgrade a database to the new format in a carefully controlled fashion. Shut down the database cleanly, then add the configuration parameter as described in the latest documentation. When starting back up, your database will be checked for compatibility with the new format and if it passes it will be upgraded, otherwise you will receive a message explaining the unsatisfied constraints.

Please note, however: while the upgrade happens in a safe manner, it is one way! So, if you request an upgrade and it succeeds, your store will be unreadable by previous versions of Neo4j.

(Un)-broken Windows

Not that Windows is broken, or that Neo4j was broken on Windows. But our tests themselves were less than fantastic. We now continuously run our full suite of quality assurance tests on both Windows and Linux before releasing to our Snapshot repository. In doing this work, we have uncovered a number of bugs that stopped our tests from running cleanly on Windows, and we have fixed them, ensuring in the process that the core of Neo4j is unproblematic.

Unfortunately, the same cannot be said about the backup component, which currently cannot perform the complete range of its operations on Windows. Until we fix it, for full backups you will need to cleanly shut down Neo4j and copy the database files. After this, you can do incremental backups, just like you’d expect.

Querious REST indexes

We added a few index-related operations to the Neo4j REST API: advanced queries and also cleaner index remove operations. Previously, you could use an exact look-up for indexed nodes or relationships. With the advanced query, any extra features exposed by a particular index framework can be queried.

For instance, using a lucene-index, let’s create a node, index it, then query:

# create an empty node
curl -X POST -H Accept:application/json https://localhost:7474/db/data/node
# create a node index named “my-nodes” backed by lucene
curl -X POST -H Accept:application/json -HContent-Type:application/json -d '{"name":"my-nodes", "config":{"type":"fulltext","provider":"lucene"}}' https://localhost:7474/db/data/index/node
# index the node (assuming the “create” step revealed node 1)
curl -X POST -H Content-Type:application/json -d '"https://localhost:7474/db/data/node/1"' https://localhost:7474/db/data/index/node/my-nodes/the_key/the_very_long_value
# query the index, expecting to find node 1 in the list
curl -H Accept:application/json "https://localhost:7474/db/data/index/node/my-nodes/the_key?query=the_very*"

To drop that node from the index, try one of the new remove operations:

# remove this way (removing node 1 with exact key/value)
curl -X DELETE -H Accept:application/json https://localhost:7474/db/data/index/node/my-nodes/the_key/the_very_long_value/1
# talk this wa-ay (removing node 1)
curl -X DELETE -H Accept:application/json https://localhost:7474/db/data/index/node/my-nodes/1

For details, read up on the REST API page.

The Little Things that Matter

We have also made a number of less dramatic changes:

  • Minor bug fixes in ShortestPath in graph-algo
  • Some bug fixes in lucene-index regarding batch insertion
  • All manpages are included in the manual
  • Added Dijkstra to the list of graph algorithms to be used when finding paths
  • Minor improvements and bugfixes to the webadmin interface
Measure Up

Ready to see how Milestone 4 measures up? Then go get it!

As always, we’re grateful to the community for contributions and feedback. Please let us know how things go with this release. Abisko isn’t far away now, so we’ll start preparing for the big arrival.

Cheers,
Andrés


Want to learn more about graph databases? Click below to get your free copy of O’Reilly’s Graph Databases ebook and discover how to use graph technologies for your application today.

Download My Ebook