Skip to main content

Managing large record sizes in Drupal and Algolia Integration

Imagine having a big homepage on our site, and when we put all the text from different fields together into 'aggregated_text_field,' the total character count goes over 10,000. Then there is a good chance that the home page won’t get indexed in Algolia as there is a 10 KB size limit per record In the free plan. Even if you switch to a paid plan, it's not a good idea to keep so much text in one record. Algolia recommends to split large records into smaller ones.

Thankfully, this is already being discussed in the issue queue of the search api Algolia module and there is a patch available. Let’s apply the patch first and see what it does.

Before we start

The patch we applied adds a search api processor that helps in splitting the large records. To configure the processor:

Algolia item splitter

configuration for Algolia aggregated text field
records splitting on Algolia dashboard

The consequence of splitting records

By splitting the records, we have created multiple records for the same content. If you visit the search page now, you will notice that the results are duplicated.

duplicate search results
Deduplication and Grouping in the Algolia dashboard
split of the original record in dashboard

dashboard result record n split

 Few more essential dashboard configurations

You have to do another crucial configuration in the dashboard. Since records are splitted, when we delete a node, all the records associated with that node should also get deleted including the splitted records. The same should happen when a node is updated as well. The patch we applied handles this but the ‘parent_record’ field should be configured as a filter from the Algolia dashboard for this to work.

essential dashboard configurations

Now, if a node is updated/deleted, it’s splits would also get updated/deleted 🎉.

We have to make one more change. Visit the search page we created and take a look at the results.

node and splits updation and deletion

The description will be cropped in each result card because we are getting the splitted record from Algolia even when there is no search term. This can be fixed by updating the js and adding a filter ‘parent_record = self’ when there is no search query. 

updaion in code

search page

We have now successfully built a fast and responsive search experience for the Umami profile using Algolia. Let’s recap what we have learned in this blog series.

As mentioned earlier in part 1 of this series, ‘search’ has become an integral part of all the modern websites. Whenever users see a search bar on your website, they expect it to be as intelligent as modern search engines. Algolia delivers this with its AI-powered capabilities and flexible APIs. I hope this blog series has given you a clear path to follow for integrating Algolia and Drupal. The complete code is available in the git repository.

We'd love to talk about your business objectives

Written by