PostgreSQL + Elasticsearch synchronization in JAVA sping (JPA)

We use JAVA Spring (JPA) and in application we write, modify and delete data in PostgreSQL.

How to synchronize data between PostgreSQL and Elasticsearch? So, they will have same data.

2 Answers

Answers 1

I assume that you are using PosgreSQL as a primary storage to store your data and Elasticsearch as a secondary storage to index the data from PosgreSQL for searching.

I would suggest Logstash is the best option. You will have create a configuration file that connects PosgreSQL with elasticsearch. Use JDBC input plugin and elasticsearch output plugin and run Logstash with that configuration file. Every document will be synced from there onward.

You also have some other options:

Batch Sync: Write an application/script (perl/python) to pull the data from PosgreSQL to index into Elasticsearch. You can schedule this job to be executed at a specific time you wish.

Real-time Sync: Right after you insert/update a record into PostgreSQL successfully, you send a request in the same API to index/update this record to Elasticsearch.

Answers 2

Here's a detailed tutorial to do that: http://david.pilato.fr/blog/2015/05/09/advanced-search-for-your-legacy-application/

If You Enjoyed This, Take 5 Seconds To Share It

• Facebook
• Twitter
• Google+
• StumbleUpon
• Digg
• Delicious
• LinkedIn
• Reddit
• Technorati