Using Jest as a REST-based Java client with ElasticSearch

If you have used ElasticSearch (ES) you will be familiar with the two ways you can access the index – the RESTful HTTP API’s and the Java API which uses a binary protocol. What is missing is a pure RESTful HTTP Java Client API. Open source Jest library attempts to fill that gap. Updated July 2016 to use ElasticSearch 2.3.4 and Jest 2.0.0.

I will assume you have some familiarity with ElasticSearch. If not please see my earlier Getting Started with ElasticSearch post.

The two ways to access ElasticSearch index are…

HTTP RESTful API. This is nothing but a set of REST endpoints you can access using one of many ways such as with CuRL or your favorite HTTP library or tools such as ElasticSearch HEAD plugin or the Chrome plugin Sense.
ElasticSearch Java API. Programmatically you can use the Java API to access the index using a very expressive and concise API. The Java API “chats” with the ES server on port 9300, whereas the RESTful HTTP client uses port 9200.

Side Note: There is another interesting option where you mix both modes. You can use the Java API’s expressive power and simplicity to build the JSON queries or index requests and then pull out the JSON string representation, turn around and submit that to the HTTP RESTful endpoints.

What ES does not have is a native library which submits requests via Java API to the RESTful endpoints (port 9200 by default). A RESTful Java API would also allow the use of a consistent load balancing strategy. Otherwise you will have to pick one way for HTTP RESTful API (maybe haproxy) and another for the native Java API Library (client node). This is assuming you use both access modes.

In comes Jest to the rescue. Jest provides its own Java API’s and also allows you to use ES Java API’s to build queries which are then submitted to the RESTful endpoints. The ability to use the ES Java API’s to build queries is greatly appreciated since I did not have to learn yet another API (#YanA).

Lets get directly to a code sample. First initialize the Jest Client. The URL provided below could point to a load balancer like haproxy too or AWS Elastic Load Balancer (ELB).

HttpClientConfig clientConfig = new HttpClientConfig.Builder("http://localhost:9200")
                                  .multiThreaded(true).build();
JestClientFactory factory = new JestClientFactory();
factory.setHttpClientConfig(clientConfig);
JestClient jestClient = factory.getObject();

HttpClientConfig clientConfig = new HttpClientConfig.Builder("http://localhost:9200")

.multiThreaded(true).build();

JestClientFactory factory = new JestClientFactory();

factory.setHttpClientConfig(clientConfig);

JestClient jestClient = factory.getObject();

Once you are done using the client, remember to shut it down using…

jestClient.shutdownClient();

1	jestClient.shutdownClient();

Next is an example of various ways to index data into the server, followed by a query. In the query example we use ElasticSearch Java Query API to help build the queries.

    private static void indexSomeData(final JestClient jestClient) throws Exception {
        // Blocking index
        final Note note1 = new Note("mthomas", "Note1: do u see this - " + System.currentTimeMillis());
        Index index = new Index.Builder(note1).index(DIARY_INDEX_NAME).type(NOTES_TYPE_NAME).build();
        jestClient.execute(index);

        // Asynch index
        final Note note2 = new Note("mthomas", "Note2: do u see this - " + System.currentTimeMillis());
        index = new Index.Builder(note2).index(DIARY_INDEX_NAME).type(NOTES_TYPE_NAME).build();
        jestClient.executeAsync(index, new JestResultHandler() {
            public void failed(Exception ex) {
            }

            public void completed(JestResult result) {
                note2.setId((String) result.getValue("_id"));
                System.out.println("completed==&gt;&gt;" + note2);
            }
        });

        // bulk index
        final Note note3 = new Note("mthomas", "Note3: do u see this - " + System.currentTimeMillis());
        final Note note4 = new Note("mthomas", "Note4: do u see this - " + System.currentTimeMillis());
        Bulk bulk = new Bulk.Builder()
                .addAction(new Index.Builder(note3).index(DIARY_INDEX_NAME).type(NOTES_TYPE_NAME).build())
                .addAction(new Index.Builder(note4).index(DIARY_INDEX_NAME).type(NOTES_TYPE_NAME).build())
                .build();
        JestResult result = jestClient.execute(bulk);

        Thread.sleep(2000);

        System.out.println(result.toString());
    }

    private static void readAllData(final JestClient jestClient) throws Exception {
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.query(QueryBuilders.termQuery("note", "see"));

        Search search = new Search.Builder(searchSourceBuilder.toString()).addIndex(DIARY_INDEX_NAME)
                .addType(NOTES_TYPE_NAME).build();
        System.out.println(searchSourceBuilder.toString());
        JestResult result = jestClient.execute(search);
        List notes = result.getSourceAsObjectList(Note.class);
        for (Note note : notes) {
            System.out.println(note);
        }
    }

private static void indexSomeData(final JestClient jestClient) throws Exception {

// Blocking index

final Note note1 = new Note("mthomas", "Note1: do u see this - " + System.currentTimeMillis());

Index index = new Index.Builder(note1).index(DIARY_INDEX_NAME).type(NOTES_TYPE_NAME).build();

jestClient.execute(index);

// Asynch index

final Note note2 = new Note("mthomas", "Note2: do u see this - " + System.currentTimeMillis());

index = new Index.Builder(note2).index(DIARY_INDEX_NAME).type(NOTES_TYPE_NAME).build();

jestClient.executeAsync(index, new JestResultHandler() {

public void failed(Exception ex) {

}

public void completed(JestResult result) {

note2.setId((String) result.getValue("_id"));

System.out.println("completed==>>" + note2);

}

});

// bulk index

final Note note3 = new Note("mthomas", "Note3: do u see this - " + System.currentTimeMillis());

final Note note4 = new Note("mthomas", "Note4: do u see this - " + System.currentTimeMillis());

Bulk bulk = new Bulk.Builder()

.addAction(new Index.Builder(note3).index(DIARY_INDEX_NAME).type(NOTES_TYPE_NAME).build())

.addAction(new Index.Builder(note4).index(DIARY_INDEX_NAME).type(NOTES_TYPE_NAME).build())

.build();

JestResult result = jestClient.execute(bulk);

Thread.sleep(2000);

System.out.println(result.toString());

}

private static void readAllData(final JestClient jestClient) throws Exception {

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();

searchSourceBuilder.query(QueryBuilders.termQuery("note", "see"));

Search search = new Search.Builder(searchSourceBuilder.toString()).addIndex(DIARY_INDEX_NAME)

.addType(NOTES_TYPE_NAME).build();

System.out.println(searchSourceBuilder.toString());

JestResult result = jestClient.execute(search);

List notes = result.getSourceAsObjectList(Note.class);

for (Note note : notes) {

System.out.println(note);

}

The full project code is on GitHub at ES-Jest Source. Review and run the RunMe.java class.

{"Mat's Random Thoughts"}

Mathew's Tech Notes..

Using Jest as a REST-based Java client with ElasticSearch