This is Fundbase Nerds, written by the team behind Fundbase.

Go to Fundbase  

Rails & Mongoid Data Loading Optimizations - Reducing Database Payload

When your queries are simple but the documents large

Posted by Marek Stanczyk on

In my previous post, I demonstrated how can you reduce the query count to make a request faster. While that’s a nice thing to do, I ran into an interesting case recently, and I’ll share my solution with you.

The scenario was that I had a list of items sorted by a non-trivial condition, which was not possible to do in a query, but had to be done in Ruby. All documents from the collection were retrieved from the database, sorted, and paginated. The problem was that the documents are rather large, and the query to load them all took long (a few seconds). This revealed a fact that having a small number of queries is not always enough.

Thinking about it, I remembered that using only can make quite a difference in the query time, but I still needed the whole documents. So I tried the following approach:

items = Model.all.only(:id, :name).entries
items.sort! { |a, b| some_custom_condition }
items = items[page_number * page_size, page_size]
items = Model.any_in(id:
items.sort! { |a, b| some_custom_condition }

So I actually ran an extra query, but the amount of data fetched from the database was decreased substantially. In my case, the request time dropped from about 7 seconds to 1.5 seconds!

Also, I was told that a good blog post should contain a picture of a cute animal, so here it is:

Marek Stanczyk

Marek is a Fullstack developer at Fundbase.
Loves beautiful code and enjoys developing with Ruby and Rails the most.