Category: ElasticSearch

ElasticSearch: High Level Client Search Scrolling

This entry is part 4 of 4 in the series ElasticSearch High Level Rest Client

In this tutorial I will show you how to perform a search scroll using the high level client. If you have not already done so please follow the search tutorial.

The reason you following the search tutorial first is that sets up the search. So you just have to do a few more steps.

Imports:

import org.elasticsearch.action.search.SearchScrollRequest;
import org.elasticsearch.common.unit.TimeValue;

Modify the “SearchRequest”. A recommended timeout is 60000 or 1m.

request.scroll(new TimeValue(60000));

Once you perform the initial search now you will get a “scrollId”. Use that to generate your new “SearchScrollRequest” using that scrollId. One thing to note is the “scrollRequest” timeout value. Set this or it may not work.

final SearchScrollRequest searchScrollRequest = new SearchScrollRequest(scrollId);
searchScrollRequest.scroll(new TimeValue(60000));

Now the searchResponse that we used initially we can repurpose to continue scrolling the results.

searchResponse = client.searchScroll(searchScrollRequest);

We know that their are no more results when the scrollId is null or when getHits length is 0.

searchResponse.getHits().getHits().length > 0

ElasticSearch: High Level Client Search

This entry is part 3 of 4 in the series ElasticSearch High Level Rest Client

In this tutorial I will show you how to perform a search using the high level client. If you have not already done so please connect to ElasticSearch.

Imports

import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.action.search.SearchType;

Now we can perform the search.

final SearchRequest request = new SearchRequest();
request.searchType(SearchType.QUERY_THEN_FETCH);

final String[] types = { "doc" };
final String[] indexes = { "index" };

//Specify the types that your search applies to.
//Note that this is not needed. If ommitted it will search all.
request.types(types);

//Specify the indexes that your search applies to.
//Note that this is not needed. If ommitted it will search all.
request.indices(indexes);

final SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
//You can add any type of query into this query. Adjust to what you need.
searchSourceBuilder.query(MyQuery);
request.source(searchSourceBuilder);

final SearchResponse searchResponse = client.search(request);

//This will let us know if the search was terminated early.
final Boolean terminatedEarly = searchResponse.isTerminatedEarly();
//This will let us know if it timed out.
final boolean timedOut = searchResponse.isTimedOut();

//Now to loop through our hits to do what we need to
final SearchHits searchHits = searchResponse.getHits();
for (final SearchHit hit : searchHits) {
  //Do work
}

 

 

 

ElasticSearch: High Level Client Post

This entry is part 2 of 4 in the series ElasticSearch High Level Rest Client

In this tutorial I will show you how to perform a POST request. If you have not connected first please do so before continuing.

Imports

import org.apache.http.HttpEntity;
import org.apache.http.nio.entity.NStringEntity;
import org.apache.http.entity.ContentType;
import org.apache.http.util.EntityUtils;
import org.elasticsearch.client.Response;

Now we can perform the POST to ElasticSearch.

final Integer id = 1;
final String document = "{\"key\": 1 }";
final HttpEntity httpEntity = new NStringEntity(document, ContentType.APPLICATION_JSON);

final Response response = restHighLevelClient.getLowLevelClient().performRequest("POST", "/indexName/indexType/" + id, Collections.<String, String>emptyMap(), httpEntity);

//Now you can print the response
System.out.println(EntityUtils.toString(response.getEntity()));

ElasticSearch: Low Level Client Get

This entry is part 3 of 3 in the series ElasticSearch Low Level Rest Client

In this tutorial I will show you how to put a json document into ElasticSearch. If you have not first connected to ElasticSearch please do so before continuing.

POM.xml

<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.9.5</version>
</dependency>

Imports

import org.apache.http.HttpEntity;
import org.apache.http.nio.entity.NStringEntity;
import org.apache.http.entity.ContentType;
import org.elasticsearch.client.Response;
import org.apache.http.util.EntityUtils;

Now perform the GET request using the low level client.

ObjectMapper objectMapper = new ObjectMapper();
final String document = "{\"key\": 1 }";
final JsonNode document = objectMapper.readTree("{" +
   " \"query\": {" +
   " \"match\" : {" +
   " \"key\" : 1 }}}");
final HttpEntity httpEntity = new NStringEntity(document.toString(), ContentType.APPLICATION_JSON);
final Response response = restClient.performRequest("GET", "/indexName/indexType/_search", Collections.<String, String>emptyMap(), httpEntity);
 
//Now you can print the response
System.out.println(EntityUtils.toString(response.getEntity()));

//OR get the content
final JsonNode content = objectMapper.readTree(response.getEntity().getContent());
System.out.println(content);

ElasticSearch: Low Level Client Put

This entry is part 2 of 3 in the series ElasticSearch Low Level Rest Client

In this tutorial I will show you how to put a json document into ElasticSearch. If you have not first connected to ElasticSearch please do so before continuing.

Imports

import org.apache.http.HttpEntity;
import org.apache.http.nio.entity.NStringEntity;
import org.elasticsearch.client.Response;
import org.apache.http.entity.ContentType;
import org.apache.http.util.EntityUtils;

Now perform the PUT request using the low level client.

final String document = "{\"key\": 1 }";
final HttpEntity httpEntity = new NStringEntity(document, ContentType.APPLICATION_JSON);
final Integer id = 1;
final Response response = restClient.performRequest("PUT", "/indexName/indexType/" + id, Collections.<String, String>emptyMap(), httpEntity);

//Now you can print the response
System.out.println(EntityUtils.toString(response.getEntity()));

ElasticSearch: High Level Rest Client Connection

This entry is part 1 of 4 in the series ElasticSearch High Level Rest Client

In this tutorial I will show you how to use the ElasticSearch high level rest client.

First you will need to add the low level rest to the pom.

<properties>
	<elasticSearch.version>6.2.4</elasticSearch.version>
</properties>

<dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-high-level-client</artifactId>
    <version>${elasticSearch.version}</version>
</dependency>

Next you will need to specify the imports.

import java.util.List;
import java.util.ArrayList;
import java.util.Arrays;
import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.elasticsearch.client.RestHighLevelClient;

Now you can connect to ElasticSearch.

final List hosts = new ArrayList<>(Arrays.asList("localhost"));
final Integer port = 9200;
final String scheme = "http";
		
final HttpHost[] httpHosts = hosts.stream().map(host -> new HttpHost(host, port, scheme)).toArray(HttpHost[]::new);

final RestClientBuilder restClientBuilder = RestClient.builder(httpHosts);
final RestHighLevelClient restHighLevelClient = new RestHighLevelClient(restClientBuilder);

Now you can do whatever you need to!

ElasticSearch: Low Level Rest Client Connection

This entry is part 1 of 3 in the series ElasticSearch Low Level Rest Client

In this tutorial I will show you how to use the ElasticSearch low level rest client.

First you will need to add the low level rest to the pom.

<properties>
	<elasticSearch.version>6.2.4</elasticSearch.version>
</properties>

<dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-client</artifactId>
    <version>${elasticSearch.version}</version>
</dependency>

Next you will need to specify the imports.

import org.apache.http.HttpHost;
import org.elasticsearch.client.Response;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;

Now you can connect to ElasticSearch.

final RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost("localhost", 9200, "http"));
 
final RestClient restClient = builder.build();

Now you can do whatever you need to!

ElasticSearch Installation

To install ElasticSearch is really straight forward. I will be using Ubuntu 16.04 for this installation.

Java 8

java -version
#if not installed run the following
sudo apt-get install openjdk-8-jdk

Download

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.3.rpm

Directories

It is recommended to change the log and data directory from default implementations.

#create log and data directory
sudo mkdir /my/dir/log/elasticsearch
sudo mkdir /my/dir/elasticsearch

# Change owner
sudo chown -R elasticsearch /my/dir/log/elasticsearch
sudo chown -R elasticsearch /my/dir/elasticsearch

Install

sudo rpm -ivh elasticsearch-6.2.3.rpm

Change Settings

sudo vi /etc/elasticsearch/elasticsearch.yml

#Change the following settings
#----------SETTINGS-----------------
cluster.name: logsearch
node.name: ##THE_HOST_NAME##
node.master: true #The node is master eligable
node.data: true #Hold data and perform data related operations
path.data: /my/dir/elasticsearch
path.logs: /my/dir/log/elasticsearch
network.host: ##THE_HOST_NAME##
http.port: 9200
discovery.zen.ping.unicast.hosts: ["##THE_HOST_NAME##"]
#----------SETTINGS-----------------

Start/Stop/Status ElasticSearch

sudo service elasticsearch start
sudo service elasticsearch stop
sudo service elasticsearch status

Rest API

http://localhost:9200