Tuesday, April 15, 2014

Neo4j 2.0.2 Maintenance Release

Today we released the 2.0.2 maintenance release of Neo4j. This release comes with some critical stability improvements as well as a few small but handy Cypher type conversion functions. All Neo4j users are strongly recommended to upgrade to this release.

Head on over to http://www.neo4j.org/download to upgrade to Neo4j 2.0.2.

Neo4j 2.0.2 does not require any store-level upgrades from the previous 2.0.1 release.

Where in the Documentation Can I Learn More about Upgrades?

The Upgrade section of the Neo4j documentation can be found here, and also covers upgrades to 2.0.

The upgrade basically happens by starting your Neo4j data store (after making the backup) with Neo4j 2.0 and the configuration allow_store_upgrade=true enabled in the configuration file conf/neo4j.properties.

To avoid running into server startup issues as part of the upgrade, you can also use the neo4j-shell for that purpose. The command line call would then be:

bin/neo4j-shell -path data/graph.db -config conf/neo4j.properties.

Information On Upgrading From Neo4j 1.9

For information about upgrading from Neo4j 1.9 or earlier, please see our Upgrading to Neo4j 2.0 FAQ.
For a full summary of changes in this release, please review the CHANGES.TXT file contained within the distribution.

For release notes for each version of Neo4j, head over to our release notes page at http://www.neo4j.org/release-notes

On behalf of the Neo4j team,

Kenny Bastani

Tuesday, March 18, 2014

Neo4j 2.0.1 Community Released on Windows Azure VM Depot

We have released a Linux distribution of Neo4j 2.0.1 community on Windows Azure's VM Depot website. Users of Windows Azure are now able to copy a platform image of Neo4j 2.0.1 directly from the VM Depot. Once provisioned, a fresh Neo4j database instance is made available via HTTP through port 7474. Check out the slides below for instructions on how to setup and provision the virtual machine image. Additionally, I wrote a separate blog post here with instructions.


For additional information about the virtual machine image, please check out the VM Depot listing at: http://vmdepot.msopentech.com/Vhd/Show?vhdId=32027&version=33072



Your feedback is always appreciated. If you run into any issues, feel free to either comment here or shout to me on Twitter: @kennybastani

Cheers,

Kenny

Friday, March 14, 2014

Spring Data Neo4j Progress Update: SDN 3 & Neo4j 2

The 3.0.1 Release

Spring Data Neo4j 3.0 was recently rolled out by the Spring Data team as part of the "Codd" release train of the Spring Data projects. We’re happy to announce this milestone, which can be used effective immediately to start developing Neo4j 2.0 applications with Spring.
Today, Spring Data Neo4j 3.0.1 containing some necessary updates was released.
The Spring Data Neo4j 3 release required a significant update of the code-base in order to support the new features and changes in Neo4j 2.0. Many of these changes included refactoring to accommodate mandatory read transactions and the removal of the reference node, as well as the required transaction-separation for schema operations. Those affected Spring Data Neo4j itself, as well as many of the dependent libraries.

We’re happy to have a first release that supports Neo4j 2.0, and that you can start developing with. We still have a few issues to work out before we recommend putting Spring Data Neo4j 3 into production, and we ask you to bear with us as we work on these last few areas. We will be releasing them as updates in the coming weeks, as soon as is possible. Please send us feedback if you run into any issues!

What has changed?

Spring Data Neo4j now works with and only with Neo4j 2.x. By default your entity types are represented as labels on the nodes and indexed simple properties are handled by Neo4j schema indexes. Queries on repositories support Neo4j 2.0 Cypher syntax and derived finders are automatically using labels, the new Cypher syntax and schema indexes.

Type Representation

A new label based type representation strategy has been added, and is now the default. Additionally, the whole inheritance hierarchy of annotated NodeEntities is actually added to each node to allow lookup on each level.
Note
The default naming for type representation has changed from the fully qualified class name to just the simple name of the class. This allows for a nicer graph model and also easier refactorings. You can still declare your own naming using the @TypeAlias annotation.

Schema Indexes

We also support schema indexes, in fact they became the default. So when you annotate your entity like this:
@NodeEntity class Person {
    @Indexed(unique=true) String ssid;
    @Indexed String name;
    int age;
    @RelatedTo(direction=BOTH) Collection<Person> friends;
}
the following indexes will be created at startup:
CREATE INDEX ON :Person(name);
CREATE CONSTRAINT ON (p:Person) ASSERT p.ssid IS UNIQUE;
Those indexes will automatically be used for appropriate queries
Note
The unique entity creation that would normally use unique constraints MERGE is not implemented yet. So if you rely on that mechanism, please configure your indexed unique fields to use the indexType=SIMPLE

Derived Finder Methods

Quite a lot of work also went into the generation of Cypher queries for derived finder methods. These now take labels and schema indexes into account.
So for the Person entity above and the following finder methods, the queries listed in the comments will be generated for this repository. Under the hood Cypher then uses schema information to take care of lookups (or label scan + property filtering).
interface PersonRepository extends GraphRepository<Person> {
	// MATCH (p) WHERE p:Person AND p.name = {name}
	List<Person> findByName(@Param("name") String name);
	// MATCH (p) WHERE p:Person AND p.age > {0}
	Collection<Person> findByAgeGreaterThan(int age);
	// MATCH (p)-[:FRIEND]-(f) WHERE p:Person and id(f) = {0}
	Set<Person> findByFriends(Person friend);
}

Legacy Indexes

Legacy indexes are still accessible, however if you want to use them you need to explicitly specify this with @Indexed(indexType=SIMPLE), same goes for numeric indexes for range queries.
Fulltext and spatial indexes also still rely on the legacy index implementations, but they were configured separately anyway.
Several of the APIs related to legacy indexes have been marked as deprecated as part of this change.

Data-Migration

At present there is no migration utility. You can upgrade your Neo4j store from 1.9 to 2.0 using the Neo4j allow_store_upgrade=true configuration setting.
Schema indexes are automatically created at startup and data of nodes with the corresponding labels is re-indexed.
The only thing that is not updated automatically is the type-representation. You might use a cypher statement like this to do it.
// superclass
START n=node:__types__(className=”com.example.domain.Person”)
SET n:Person
WHERE n.__type__ = ”com.example.domain.Person”
SET n:_Person

// subclass
START n=node:__types__(className=”com.example.domain.Employee”)
SET n:Employee:_Employee
For testing Spring Data Neo4j 3.0.1 and providing feedback it shouldn’t be a problem to import some data into it.
For the next minor release we’ll work on providing a migration tool though.

Setup & Configuration

As usual, you add Spring Data Neo4j 3.0 as a dependency to your pom.xml
<dependency>
    <groupId>org.springframework.data</groupId>
    <artifactId>spring-data-neo4j</artifactId>
    <version>3.0.0.RELEASE</version>
</dependency>
Neo4j 2.0 introduced the requirement separately manage schema and data transactions. We tried several approaches to handle this automatically, none of which worked in a satisfactory manner. This has resulted in a few breaking changes which needed to be introduced in order to cater for this and is detailed below.
Important
So we had to make the entity metadata handling an explicit step in the lifecycle and decided to change the base-package attribute for neo4j:config to be mandatory. It allows for one or more (comma separated) package names to be scanned for entities.
<neo4j:config storeDir="graph.db" base-package="com.example.domain"/>
<neo4j:repositories base-package="com.example.repositories"/>
Or in your @Configuration (which will change to a nicer way soon, tacked to @EnableNeo4jRepositories):
@Configuration
@EnableNeo4jRepositories(basePackage="com.example.repositories")
static class TestConfig extends Neo4jConfiguration {
    TestConfig() {
        setBasePackage("com.example.domain");
    }
Looking forward to your feedback,

Nicki and Michael

If you are interested, you can also check out our presentation at GraphConnect that discussed the intermediary state of this work:

Object Graph Mapping with Spring Data Neo4j 3 - Nicki Watt & Michael Hunger @ GraphConnect London 2013 from Neo Technology on Vimeo.

Friday, February 28, 2014

Blog Post: Community Support

First of all, I want to say how happy I am to be part of such a great community around Neo4j.

What makes the Neo4j community so impressive is not just the fact that many users apply graphs in so different contexts, but especially the supporters of the community who help you and us by jumping in, answering questions, offering advice and solving problems.

People like Wes Freeman, Luanne Misquitta, Johannes Mockenhaupt, Nicholas Stuart, Aravind Yarram, Jonatan J├Ąderberg, Lisa Li, Ulkas, Tatham Oddie, Nigel Small and all the others really make the difference and we are very thankful for that.

And so can you! It is really the small gestures that create a community. Imagine you've just been helped with a tricky problem, and a new question pops up that you already know the answer for. Then it is a quick thing for you to share your advice and help the next poster to become successful with Neo4j too.

And no one has to overdo it. Once a week is great. If enough people do that it will be a huge help for everyone.

Doing so on Stackoverflow is even more rewarding, besides getting the rare Neo4j badges. You will grow your experience with Neo4j, it will show publicly and this could be an important aspect for your CV. If you are with a Neo4j partner, having strong Neo4j skills makes it more likely for customers to contract you.


SO_Badges.png

A great way of getting started is to just follow @neoquestions (courtesy Wes Freeman) which tweets StackOverflow questions about Neo4j.

Another great variant that Mark, Max and Stefan use quite often, is just to blog about the things you’ve learned. Whenever you learned something new or solved a problem, write a quick blog post about it. Then google will always find it for you (and others) if you ever encounter it again. And after a posting make sure to tweet it out, tagged #neo4j, so that we can help promoting it.


We really love to support you in a timely and exhaustive manner. But the rising popularity of Neo4j also leads to a steep increase in questions on both channels, answering which take more and more time of our long days.


That's why we wanted to make sure that the expectations you have for Community Support are aligned with what we can deliver.


Community support is a free offering with no guarantees or promises. We can't promise to solve your question or answer in a reasonable time. We will also not be able to do one-to-one conversations, all questions and answers have to be in public, so that everyone benefits.


We also can't support you through through proof-of-concept implementations that are more involved or complex modeling questions.


Where should I ask what?


  1. For things that don’t work and code-related questions, please use StackOverflow and tag your questions with neo4j, cypher, etc (e.g "neo4jclient", "py2neo").
  2. The Google Group is more of a discussion forum about features, graph modeling and generally applicable questions about Neo4j and projects around it.
  3. For real bugs please raise a GitHub issue.


If you have need for dedicated, commercial support around Neo4j don't hesitate to ask for it. Our local partners and also our sales-team is happy to provide you with focused attention and guaranteed response time for your challenges. We also offer consulting and on-site workshops. Some of our helpful community members also offer one-to-one time on services like Wes.


To help you efficiently, you have to help us as well. If you submit a Neo4j question to StackOverflow or the Google Group, make sure it has all the information needed to analyse it and make sensible suggestions:


Make sure:


  • you have searched for your problem before asking on Google, StackOverflow, GitHub issues or the Google Group
  • to reproduce your issues with the latest stable versions of Neo4j, perhaps it has already been resolved, and is not a problem anymore
  • all the version information available on Neo4j, Java, OS, any libraries or tools you use
  • you can provide a good explanation and picture of your graph model, including number of nodes, types, relationships per type, etc.
  • you provide the usual log-files which contain a lot of diagnostics information: /path/to/neo/data/graph.db/messages.log and /path/to/neo/data/log/console.log
  • You provide enough information about the code you use to recreate the situation at hand: source code, test-projects with unit-tests (preferred), Cypher queries, configuration, REST-calls
  • your Cypher queries are easily understandable with sensible variable names and well formatted
  • It helps a lot if you can make your Cypher Questions available with a dataset to try them on, please use a GraphGist or the Neo4j Console


If your database doesn't contain sensitive data, it helps a lot to zip it up and make it available on DropBox or GoogleDrive.


So thanks again for being such a great community, let’s rock it together!


Cheers


Michael for the Neo4j Team

Wednesday, February 26, 2014

Graph Gist Winter Challenge Winners

To be honest, we were blown away.



When starting this challenge we were really excited and curious about the results. But what YOU created and submitted is just impressive.


We received 65 submissions in the 10+ categories. Well done!


Make sure to check them out, each one is a jewel on its own and there are many surprises hidden in these submissions. And if you get started with Neo4j one of these domains you might already have your modeling and use-case work halfway done. So before starting a proof of concept project, have a look.


You can certainly imagine, it was really hard for us to choose the winners (for sheer volume and quality). The quality of the submissions is really astonishing and we hope it will get even better when the feedback from the commenting sections is taken into account.


Everyone who participated will receive a Neo4j T-Shirt (if postal address and size was submitted here) and the winners will get an Amazon gift certificate (300,150,50 USD).


But without further ado let’s look at the categories and the winners:

Education

  1. Organization Learning by @luannem - covering your path through courses and certifications in a learning management system.
  2. Degrees offered by the University of Oviedo by @leyvanegri - solving use-cases for students at a university.
  3. Interpreting Citation Patterns in Academic Publications: A research aid by Jonatan J├Ąderberg - an advanced use of graphs to connect scientific papers.

Finance

  1. Graphing our way through the ICIJ offshore jurisdiction data by @hermansm - an impressive investigative tracking of leaked data sets about (legal) company activities.
  2. Finance and Asset Management by @rushugroup is an interesting set of financial portfolio analytics use-cases.
  3. Options Trading As A Graph by @lyonwj looks at how to model the tricky business of option trading in a graphy way.
Foundation4Gist.png

Life Science

  1. Medicine & drugs classification for the Central Hospital of Asturias by @Roqueeeeee and @luigi9215 is an impressive representation of drug-related use-cases for a hospital.
  2. Competitive Intelligence in Cancer Drug Discovery by @livedataconcept cleanly models and queries available cancer drugs.
  3. DoctorFinder! by @fbiville & the VIDAL team is a real life application on how to find the drugs and doctors for your symptoms.

Manufacturing

  1. Project Management by @_nicolemargaret shows how graphs are perfect for dependency management in an incremental fashion.
  2. Car Manufacturers 2013 by @fernanvic1 explores the intricate network of car manufacturers, their brands, investments and models.
  3. Device manufacture trends by @shantaramw let’s you glimpse on how graphs can also exploited for business intelligence use-cases.

Sports

  1. Alpine Skiing seasons by @pac_19 uses an intricate model to map the real FIS data into the graph to find some really cool insights.
  2. F1 2012/2013 Season by @el_astur answers many different questions by looking at Formula one racing data.
  3. League of Legends eSports - LCS by @SurrealAnalysis looks at different analytical statistics of the League Championship Series.
Visio.png

Resources

  1. EPublishing: A graphical approach to digital publications by @deepeshk79 impressively covering a lot of different use-cases in the publication domain and workflow.
  2. Piping Water by @shaundaley1 looks at London’s pipe system and how that natural graph could be managed by using a graph database.
  3. QLAMRE: Quick Look at Mainstream Renewables Energies by @Sergio_Gijon is a quick look at categorizations of renewable energies.


The Antarctic Research: The Effect of Funding & Social Connections in the US Antarctic Program by @openantarctica is really impressive but sadly not eligible as the demo dataset used is too large for the limited scope.

Retail

  1. Food Recommendation by @gromajus uses a graph model of food, ingredients and recipes to compute recommendations, taking preferences and allergies into account.
  2. Single Malt Scotch Whisky by @patbaumgartner is my personal favorite, you certainly know why :) Ardbeg 17 is the best.
  3. Phone store by @xun91 uses phone models, attributes, manufacturers and stock information to make recommendations for customers.

Telecommunication

  1. Amazon Web Services Global Infrastructure Graph by @AIDANJCASEY represents all regions, zone, services and instance types as a graph awesome for just browsing or finding the best or cheapest offering.
  2. Geoptima Event Log Collection Data Management by @craigtaverner is a really involved but real world model of mobile network event and device data tracking.
  3. Mobile Operators in India by @rushugroup is a basic graph gist exploring the Indian phone network by device technology and operators.

sandakphu_map.jpg

Transport

Transport and routing is a great domain for graphs and we see a lot of potential here, unfortunately the sandbox is not well suited for the some of the large demo datasets, so some of the entries did not qualify.


  1. Roads, Nodes and Automobiles by @tekiegirl shows how user provided road maps could be represented in a graph and what can you do with it. There are great example queries for the M3 and M25 motorways in the UK.
  2. Bombay Railway Routes by @luannem shows advanced routing queries for the infamous railway network.
  3. Trekking and Mountaineering routing by @shantaramw Himalayan routes in a graph are not just for hard-core trekkers and bikers, with useful answers.


Advanced Graph Gists

As expected this has been most impressive, people really went far and wide to show what’s possible with graphs and graph-gists. Really hard to choose in this category.



  1. Movie Recommendations with k-NN and Cosine Similarity by @_nicolemargaret Nicole really shows off, computing, storing and using similarities between people for movie rating.
  2. Skip Lists in Cypher by @wefreema - a graph is a universal data structure, why not use it for other data structures too. Wes shows how with a full blown skip list implementation with Cypher.
  3. Small Social Networking Website by @RaulEstrada this is not over the top like others but a really good and comprehensive example on what graphs are good for.
drinks.png

Other

This category unintentionally sneaked in but had some really good submissions. So we also award some prizes here. It’s like the little brother of the Advanced category.


  1. Embedded Metamodel Subgraphs in the FactMiners Social-Game Ecosystem Part 2 by @Jim_Salmons explores the possibilities of using data and meta-data in the same graph structure and which additional information you can infer about your data.
  2. Legislative System Graph by @yaravind is an impressive collection of use cases on top of electorate data.
  3. User, Functions, Applications, or "Slicing onion with an axe" by @karol_brejna covers resource and permission management of an IT infrastructure.


I not only want to thank all of you who contributed, but also our awesome judging team (Mark, Wes, Luanne, Jim, Kenny, Anders, Chris) who spent a lot of time looking at the individual GraphGists and provided valuable feedback in the comment sections. So please authors thank them by updating your gists and taking those comments into account!


As we want you to always publish your awesome graph models, we’d like you to know:


Everyone who, now or in the future, submits a GraphGist on a new topic
via this form will get a t-shirt from us.



GraphGists are a great initial graph model for anyone starting with Graphs and Neo4j. 
That’s why we want you to vote on the gists your really like or found helpful.
Thank you!


In case you wonder what the “Rules for a good GraphGist” are, that we used for judging, here are some of them. So if you work on a GraphGist in the future, please keep them in mind:
  • interesting/insightful domain
  • a good number of realistic use-cases with sensible result output
  • description, model picture should be easy to understand
  • sensible dataset size (at most 150 nodes 300 rels)
  • good use of the GraphGist tools (table, graph, hide-setup etc)
  • we had an epiphany while looking at the gist


And last but not least a special treat. The structr team has added GraphGist import to structr so you can automatically create a schema and import the initial dataset into your graph-based application. Then add some use-case endpoints and you’re done.

Michael for the Neo4j Team