Elasticsearch: Searching nested documents

When it comes to dealing with big volumes of data, elasticsearch performs really well. It can store, search and analyze data in near real time. Most of the times the schema of such a large data set is not simple, but complex. This blog provides a solution for performing search within a data set with such complex schema. 

Goal 

To match elements of an array and fetch them in sorted order.

Note: We only require the matched elements of the array, not the parent document.  

Problem

If we look closely at our goal, we can divide it into two parts:-

1. Fetch the matching elements of the array.
2. Sort them based on some factor, let’s say relevance.

First part of the solution is pretty straight forward. This can be achieved using a simple Nested query. The problem arises in the latter part. We cannot use a nested query to bring the matched document in required order. 

Solution

With a simple nested query we get the results based on relevance of the whole document matched rather than the document inside the array.

In order to get the elements based on relevance, we can use a combination of different types of aggregations in elasticsearch.

{
  "size": 0,
  "aggs": {
    "aggs": {
      "filter": { /your search query/ },
      "Aggregation_Name": {
        "nested": {
          "path": "array_name"
        },
        "aggs": {
          "result": {
            "top_hits": {
              "size": 10
            }
          }
        }
      }
    }
  }
}

This query will fetch the result based on the matching elements inside array, treating every element as a seperate document,(and ignoring the parent document), in the desired order.