Pages

Liquigraph as a data migration tool for Neo4j

I have been working a graph database recently called Neo4j. Its interesting. Since its a graph database, you have to think differently. And so, it might be intimidating for those who are used to the sql databases. Especially because they are familiar with how it works and all the tools required to work with it. I was particularly focused on Application database migration tools.

Directly to the point:
There are a couple of tools for automatic data migration for sql databases

  • Liquibase
  • Flyway
But since we are also using Neo4j (graph database), above migration tools won't work.
Well I found a tool called 
  • LIQUIGRAPH
Its basically similar to Liquibase. 

How it works?

Basically most of the getting started tutorial is here
  • Add the dependency in the spring project. 
  • Configure the datasource. (Tricky is multiple databases)
  • Add migration scripts in changelog.xml at your resources/db/liquigraph directory
Now the new changes logged in the changelog.xml will be applied to the graph data. 

Will it apply the change on every application start?

No. It creates some nodes in your graph database (_liquigraphchangelog, _liquigraphchangeset, _liquigraphquery). These are to track the previous migrations. Liquigraph knows which changes are already applied to the graph and which are the new changes, and applies only the new changes.

What are the dependencies?

Add the two dependencies

implementation ('org.liquigraph:liquigraph-spring-boot-starter:3.1.1')
implementation ('org.neo4j:neo4j-jdbc-bolt:3.5.1')

Add the properties in the properties file

liquigraph:
  user: neo4j
  password: pswd
  url: jdbc:neo4j:bolt://localhost:7687

Put the changes in the changelog.xml
<?xml version="1.0" encoding="UTF-8"?><changelog xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"           xsi:noNamespaceSchemaLocation="http://www.liquigraph.org/schema/1.0/liquigraph.xsd">
    <changeset id="hello-world" author="you">
        <query>CREATE (n:Sentence {text:'Hello monde!'}) RETURN n</query>
    </changeset>
    <changeset id="hello-world-fixed" author="you">
        <query>MATCH (n:Sentence {text:'Hello monde!'}) SET n.text='Hello world!' RETURN n</query>
    </changeset>
    <changeset id="set-author-in-sentence" author="you">
        <query>match(n: Sentence) set n.author= 'rajan' return count(n)</query>
    </changeset>
</changelog>

Now if these changes are not already applied in the graph, They will be applied at the application restart.

Problems/Confusions 

  • I was confused about specifying the liquigraph properties explicitly. I thought since I have already configured the spring.data.neo4j properties, it would pick those properties to configure the Datasource. But it does not.
  • The tutorial tells you to put spring-data-jdbc to the dependencies. I was afraid that it would mess with spring-data-jpa that I had in my project. So the second way of creating a Datasource manually won't work either. There were miltiple databases in my project (MSSQL, Mongodb, Neo4j). So faced some issue as to which Datasource the Liquigraph was picking. I had to dig inside the library to find out that the liquigraph property could be the solution, and it worked.
  • Pick proper version of the libraries

7 comments:

  1. Very well written article. It was an awesome article to read. Complete rich content and fully informative. I totally Loved it. Azure Data Migration Services

    ReplyDelete
  2. I always check these types of advisory posts and I found your article. This is a great source to increase knowledge. Thanks for sharing an article like this. Philippines Customs Data

    ReplyDelete
  3. best Vietnam import data
    we are gives best quality services in very low prices

    ReplyDelete
  4. one of the best US import data
    We Are give best quality services in vey low prices

    ReplyDelete
  5. You've created an excellent post. Your essay gave me some unique and useful information. Thank you for bringing this article to our attention. SAP integration software

    ReplyDelete

If you like to say anything (good/bad), Please do not hesitate...