0

My Linux distribution uses the redis database. At boot the redis-server needs about 80 seconds to load the dataset. The following is a log that shows what I have written:

redis-server[249]: 249:M 17 Oct 2022 16:29:55.173 * DB loaded from append only file: 79.442 seconds

If a Python program tries querying the database before the redis-server has finished the loading in memory operation, it is raised the Exception: redis.exceptions.BusyLoadingError.
The Exception description is: Redis is loading the dataset in memory and is compliant with the context that I have described (that is the database is loading data in memory).

Because I'm using the default configuration of redis-server in this moment I don't know exactly what is the type of persistence used by redis-server.
The file redis.conf is very long so, here, I report settings that I think are most important:

...
########################### SNAPSHOTTING  ###########################
#
# Save the DB on disk:
#
#   save <seconds> <changes>
#
#   Will save the DB if both the given number of seconds and the given
#   number of write operations against the DB occurred.

# OE: tune for a small embedded system with a limited # of keys.
save 120 1
save 60 100
save 30 1000

######################### APPEND ONLY MODE #########################
# OE: changed default to enable this
appendonly yes

# The name of the append only file (default: "appendonly.aof")
appendfilename "appendonly.aof"
...

These settings (in particular appendonly yes) seem indicate that the persistence type used by the database is: Append Only File (AOF). I think these settings are causing this long loading time.

Is it possible to use settings that avoid a too long loading time at boot?

1 Answer 1

1

This is very likely because your redis-server process is configured to use the append-only file, and you never sent the BGREWRITEAOF command.

The Redis Manual on the redis.io website has a good writeup of the four different persistence (data recovery) configurations: https://redis.io/docs/manual/persistence/. The append-only file is merely one of the options.

It's not a bad option, it simply requires a little maintenance, or else, as you have discovered, it makes the process start slowly. Read about the options and their benefits/drawbacks and pick the one that's best for your usage patterns.

4
  • thanks. I'll study your suggestions and I'll give you a feedback. Any other pratical hints are accepted!
    – User051209
    Oct 18, 2022 at 6:52
  • Thanks, with BGREWRITEAOF the time of loading data in memory has been greatly reduced.
    – User051209
    Oct 21, 2022 at 17:12
  • I have found this useful link which explains how to use the BGREWRITEAOF command.
    – User051209
    Oct 24, 2022 at 15:39
  • Yes, if you read the persistence doc page I liked to in my answer, farther down the page is a description of some auto-aof-rewrite-percentage settings. And there's the source doc page for the BGREWRITEAOF command itself: redis.io/commands/bgrewriteaof
    – Sotto Voce
    Oct 24, 2022 at 17:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .