1. Overview
In Elasticsearch, an index is an important data structure that decides how data is stored, organized, and retrieved. So, listing down the indices sorted by name is a common operation involved in the regular monitoring and maintenance of the Elasticsearch cluster.
In this tutorial, we’ll learn how to list the indices using the REST APIs exposed by Elasticsearch and the Kibana interface.
2. Scenario Setup
In this section, let’s bootstrap an Elasticsearch cluster locally to solve the use case of listing indices.
2.1. Spinning Up Elasticsearch
Let’s start by using the docker run command to start a single node Elasticsearch cluster:
$ docker run -d --name elasticsearch \
-p 9200:9200 -p 9300:9300 \
-e "discovery.type=single-node" \
docker.elastic.co/elasticsearch/elasticsearch:7.13.2
We exposed the 9200 and 9300 ports for network communication with the cluster. We’ll use the 9200 port with all our REST APIs.
After waiting for a few seconds, we can use the health API to ensure that our Elasticsearch cluster has a green status:
$ curl -s -X GET 'http://localhost:9200/_cluster/health' | jq -r '.status'
green
We used the jq command to extract the status field from the larger JSON response.
2.2. Generating Sample Indices
Let’s use the index API to create three sample indices to simulate and solve our use case of listing the indices sorted by name.
First, we can create small-index with 5 documents:
$ for i in {1..5}
do
curl -X POST "localhost:9200/small-index/_doc/$i" -H 'Content-Type: application/json' -d'
{
"field1": "small-document '"$i"'",
"field2": "value'"$i"'"
}'
done
We used a for loop to call the POST /
Next, let’s create medium-index by adding 15 documents to it:
$ for i in {1..15}
do
curl -X POST "localhost:9200/medium-index/_doc/$i" -H 'Content-Type: application/json' -d'
{
"field1": "medium-document '"$i"'",
"field2": "value'"$i"'"
}'
done
We must note that the document’s content doesn’t matter for our use case.
Further, let’s create large-index with 50 documents:
$ for i in {1..50}
do
curl -X POST "localhost:9200/large-index/_doc/$i" -H 'Content-Type: application/json' -d'
{
"field1": "large-document '"$i"'",
"field2": "value'"$i"'"
}'
done
Lastly, let’s create aliases for each of these indices, as it’s a common practice to use aliases while working with indices:
$ curl -X PUT "localhost:9200/small-index/_alias/index1"
{"acknowledged":true}
$ curl -X PUT "localhost:9200/medium-index/_alias/index2"
{"acknowledged":true}
$ curl -X PUT "localhost:9200/large-index/_alias/index3"
{"acknowledged":true}
Great! We’re all set to explore different solutions for our use case.
3. Using the cat indices API
We can use the cat indices API to retrieve information about the indices in a human-readable tabular format. Further, we can use query parameters to fine-tune its output.
Firstly, let’s use the API with query parameters, such as v to enable column headings and h to specify the columns to display:
$ curl -X GET "localhost:9200/_cat/indices?pretty&v&h=index,docs.count,store.size"
index docs.count store.size
small-index 5 5kb
large-index 50 7.6kb
medium-index 15 6kb
By default, the output isn’t sorted by indices name.
Now, let’s add the s, query parameter to sort the response by the index column:
$ curl -X GET "localhost:9200/_cat/indices?pretty&v&h=index,docs.count,store.size&s=index"
index docs.count store.size
large-index 50 7.6kb
medium-index 15 6kb
small-index 5 5kb
The indices are sorted by name in an ascending order.
Lastly, we can add the :desc suffix to the column name to sort the response in descending order:
$ curl -X GET "localhost:9200/_cat/indices?pretty&v&h=index,docs.count,store.size&s=index:desc"
index docs.count store.size
small-index 5 5kb
medium-index 15 6kb
large-index 50 7.6kb
The result looks as expected. However, this approach is not recommended for applications or automation scripts. So, let’s continue exploring a few more effective ways to solve our use case.
4. Using the Get alias API
As the name suggests, the Get alias API retrieves the indices with their aliases. We can use this API in automation scripts to get indices’ names along with their aliases, as the cat indices API doesn’t provide information on aliases.
Let’s use the alias API on our Elasticsearch cluster:
$ curl -s -X GET "localhost:9200/_aliases?pretty"
{
"large-index" : {
"aliases" : {
"index3" : { }
}
},
"small-index" : {
"aliases" : {
"index1" : { }
}
},
"medium-index" : {
"aliases" : {
"index2" : { }
}
}
}
The response includes a mapping of indices with their aliases.
Now, let’s use jq to extract the indices’ names and aliases and then sort them by names with the sort command:
$ curl -s -X GET "localhost:9200/_aliases" | \
jq -r 'to_entries[] | "\(.key) \((.value.aliases | keys[]))"' | sort
large-index index3
medium-index index2
small-index index1
Great! The indices are now sorted by name in ascending order. We can also use the -r flag with sort to list indices in descending order.
Additionally, let’s use the -k flag to sort the indices by aliases available in the 2nd column:
$ curl -s -X GET "localhost:9200/_aliases" | \
jq -r 'to_entries[] | "\(.key) \((.value.aliases | keys[]))"' | sort -k2
small-index index1
medium-index index2
large-index index3
The output looks as expected.
5. Using Index stats API
While managing an Elasticsearch cluster, we might want to list indices’ names and UUIDs for troubleshooting. For such a scenario, we can use the stats API to retrieve detailed statistics about the indices.
Let’s use the stats API to list the indices sorted by names in ascending order along with their UUID:
$ curl -s -X GET "localhost:9200/_stats?pretty" | \
jq -r '.indices | to_entries[] | "\(.key) \(.value.uuid)"' | sort
large-index _BaHhU-NSNu5YahOZ07qIg
medium-index vZL_wZJqRhGpjY0L7xKGng
small-index dUu-1KERTnGJAi47KOLSLQ
Since the entire API response is too verbose, we used jq to extract the name and UUID of indices through the .indices JSON path. Further, we used the to_entries[] function to map the indices objects into an array of key-value pairs to extract the fields. At the end, we piped the output with sort.
Similarly, we can list the indices sorted by names in descending order by using the -r flag with sort:
$ curl -s -X GET "localhost:9200/_stats" | \
jq -r '.indices | to_entries[] | "\(.key) \(.value.uuid)"' | sort -r
small-index dUu-1KERTnGJAi47KOLSLQ
medium-index vZL_wZJqRhGpjY0L7xKGng
large-index _BaHhU-NSNu5YahOZ07qIg
Perfect! We’ve got the desired output.
6. Using Cluster state API
While the alias API doesn’t provide UUIDs, the stats API doesn’t show the alias information. If we want to get these details together, we can use the cluster state API to get a comprehensive view of the Elasticsearch cluster and its indices.
Again, the response for this API is quite lengthy, so we’ll use jq to extract the indices’ names, UUIDs, and aliases. Let’s list the indices sorted in ascending order by their name:
$ curl -s -X GET "localhost:9200/_cluster/state/metadata" \ | \
jq -r '.metadata.indices | to_entries[] | "\(.key) \(.value.aliases| join(", ")) \(.value.settings.index.uuid)"' | sort
large-index index3 _BaHhU-NSNu5YahOZ07qIg
medium-index index2 vZL_wZJqRhGpjY0L7xKGng
small-index index1 dUu-1KERTnGJAi47KOLSLQ
We must note that the indices’ names are available through the .metadata.indices JSON path. Further, we used to_entries[] to convert the indices object into an array of key-value pairs for iteration. Lastly, we constructed a string using the name, aliases, and UUID for each index.
Like earlier, we can list the indices sorted in descending order using the -r flag with sort.
7. With Kibana
In this section, we’ll list the indices sorted by names using Kibana’s convenient and user-friendly GUI interfaces.
7.1. Spinning Up Kibana
For our use case, we can spin up Kibana and expose its user interface at the 5601:
$ docker run -d --name kibana \
--link elasticsearch:elasticsearch \
-p 5601:5601 \
-e ELASTICSEARCH_HOSTS=http://elasticsearch:9200 \
docker.elastic.co/kibana/kibana:7.13.2
To access the GUI, we can visit http://localhost:5601 in a browser.
7.2. Using the Index Management View
Let’s access the index management view in Kibana to retrieve the list of indices:
http://localhost:5601/app/management/data/index_management/indices
Here, we can view information about indices.
We can see that it’s super intuitive to use and sort the indices by clicking on any of the headers:
We can click on the first column to sort the list by name in ascending or descending order.
7.3. Using Dev Tools Console
Alternatively, we can open the dev tools console to interact with the ElasticSearch cluster:
http://localhost:5601/app/dev_tools#/console
Here, we can enter and execute ElasticSearch queries.
Let’s see the cat indices API in action:
We got the output containing the list of indices sorted by name. Additionally, we must note that we added /*-index in the query path to filter the list to contain only the indices that interest us.
8. Conclusion
In this article, we learned how to list the indices in Elasticsearch sorted by name. Moreover, we explored different REST APIs, such as the cat indices API, alias API, index stat API, and cluster state API, to solve the use case.
Lastly, we used Kibana’s index management view and dev tools console to list the indices.