[BUG] Physical DB Migration From Redis 7.2.5 To Valkey 8.1.0 Ineffective

by ADMIN 73 views

Describe the Bug

In this article, we will discuss a bug that occurred during a physical database migration from Redis 7.2.5 to Valkey 8.1.0. The migration process failed to retain the contents of a dump.rdb file created by Redis 7.2.5. This bug is significant because it highlights the importance of proper migration procedures when upgrading from one database system to another.

To Reproduce

To reproduce this bug, follow these steps:

  1. Set up a bare-metal Redis 7.2.5 instance on Arch Linux.
  2. Migrate the Redis instance to a bare-metal Valkey 8.1.0 instance on Arch Linux, as per the "Physical migration" section of the migration documentation.
  3. Ensure that the Valkey configuration has the append-only-file (AOF) functionality commented out before the first start, as instructed by the documentation.
  4. After the first attempt fails, make a second attempt by not only disabling AOF but also deleting the appendonlydir directory before the first Valkey start.

Expected Behavior

The expected behavior is that Valkey continues the Redis database state, retaining the contents of the dump.rdb file created by Redis 7.2.5.

Actual Behavior

However, the actual behavior is that Valkey resets the database or starts a new database, losing the contents of the dump.rdb file.

Additional Information

The Arch Linux packages used in this migration are:

  • redis 7.2.5-1
  • valkey 8.1.0-3

A command-line session excerpt is provided below to illustrate the issue:

# Examining the Redis instance as instructed by the migration guide
[paperless@the-system paperless]$ redis-cli -s /srv/paperless/redis/redis.sock 

redis /srv/paperless/redis/redis.sock> INFO KEYSPACE
# Keyspace
db0:keys=117,expires=115,avg_ttl=784735

redis /srv/paperless/redis/redis.sock> CONFIG GET dir dbfilename
1) "dbfilename"
2) "dump.rdb"
3) "dir"
4) "/srv/paperless/redis"

redis /srv/paperless/redis/redis.sock> SAVE
OK

redis /srv/paperless/redis/redis.sock> exit


# State of the Redis database directory before the first Valkey start
# on attempt 1 (Redis is not running)
[paperless@the-system paperless]$ find redis | sort                                                                                                          
redis
redis/appendonlydir
redis/appendonlydir/appendonly.aof.7.base.rdb
redis/appendonlydir/appendonly.aof.7.incr.aof
redis/appendonlydir/appendonly.aof.manifest
redis/dump.rdb
redis/redis.conf


# The Redis database directory was cloned from "redis" to "kvdb", and the UNIX
# socket was changed from "redis/redis.sock" to "kvdb/kvdb.sock"
# Valkey configuration was copied from the Redis configuration and edited
# accordingly

# Valkey was started with these keys commented out in its configuration:
#  - appendonly
#  - appendfilename
#  - appendfsync
[paperless@the-system paperless]$ valkey-cli -s /srv/paperless/kvdb/kvdb.sock 

valkey /srv/paperless/kvdb/kvdb.sock> INFO KEYSPACE
# Keyspace
db0:keys=2,expires=0,avg_ttl=0

valkey /srv/paperless/kvdb/kvdb.sock> CONFIG GET dir dbfilename
1) "dbfilename"
2) "dump.rdb"
3) "dir"
4) "/srv/paperless/kvdb"

Valkey Configuration

The Valkey configuration used in this migration is as follows:

# Disable TCP socket listening completely, an UNIX socket is used instead
# If port 0 is specified Valkey will not listen on a TCP socket
port                 0
protected-mode       yes

# Listen on UNIX socket
# Valkey URL: unix:///srv/paperless/kvdb/kvdb.sock
unixsocket           /srv/paperless/kvdb/kvdb.sock
unixsocketperm       660

# Server behavior
timeout              0
daemonize            no
supervised           systemd
pidfile              /srv/paperless/kvdb/kvdb.pid
loglevel             warning
logfile              ""
databases            1
always-show-logo     no
set-proc-title       yes
proc-title-template  "{title} paperless-dedicated instance"
# Working directory; database files go here
dir                  /srv/paperless/kvdb/
# Default database file
dbfilename           dump.rdb
# Persist the DB after 300 seconds (5 minutes) if at least 1 key has changed
save                 300 1
## Write an additional append-only database file for increased crash resilience
#appendonly           yes
#appendfilename       appendonly.aof
#appendfsync          everysec

Conclusion

In conclusion, the physical database migration from Redis 7.2.5 to Valkey 8.1.0 was ineffective due to the failure to retain the contents of the dump.rdb file created by Redis 7.2.5. This bug highlights the importance of proper migration procedures when upgrading from one database system to another. By following the steps outlined in this article, developers can reproduce this bug and take necessary precautions to avoid similar issues in their own projects.

Recommendations

To avoid this bug, we recommend the following:

  1. Carefully review the migration documentation for Valkey 8.1.0 to ensure that all necessary steps are taken to preserve the contents of the dump.rdb file.
  2. Verify that the Valkey configuration has the append-only-file (AOF) functionality commented out before the first start.
  3. Consider using a different migration strategy, such as a logical migration, to minimize the risk of data loss.
  4. Regularly back up the Redis database before attempting a migration to ensure that data can be recovered in case of an issue.

Q: What is the issue with the physical database migration from Redis 7.2.5 to Valkey 8.1.0?

A: The issue is that the migration process failed to retain the contents of the dump.rdb file created by Redis 7.2.5. This resulted in Valkey resetting the database or starting a new database, losing the contents of the dump.rdb file.

Q: What are the steps to reproduce this bug?

A: To reproduce this bug, follow these steps:

  1. Set up a bare-metal Redis 7.2.5 instance on Arch Linux.
  2. Migrate the Redis instance to a bare-metal Valkey 8.1.0 instance on Arch Linux, as per the "Physical migration" section of the migration documentation.
  3. Ensure that the Valkey configuration has the append-only-file (AOF) functionality commented out before the first start, as instructed by the documentation.
  4. After the first attempt fails, make a second attempt by not only disabling AOF but also deleting the appendonlydir directory before the first Valkey start.

Q: What is the expected behavior of Valkey during the migration process?

A: The expected behavior is that Valkey continues the Redis database state, retaining the contents of the dump.rdb file created by Redis 7.2.5.

Q: What is the actual behavior of Valkey during the migration process?

A: The actual behavior is that Valkey resets the database or starts a new database, losing the contents of the dump.rdb file.

Q: What are the Arch Linux packages used in this migration?

A: The Arch Linux packages used in this migration are:

  • redis 7.2.5-1
  • valkey 8.1.0-3

Q: What is the Valkey configuration used in this migration?

A: The Valkey configuration used in this migration is as follows:

# Disable TCP socket listening completely, an UNIX socket is used instead
# If port 0 is specified Valkey will not listen on a TCP socket
port                 0
protected-mode       yes

# Listen on UNIX socket
# Valkey URL: unix:///srv/paperless/kvdb/kvdb.sock
unixsocket           /srv/paperless/kvdb/kvdb.sock
unixsocketperm       660

# Server behavior
timeout              0
daemonize            no
supervised           systemd
pidfile              /srv/paperless/kvdb/kvdb.pid
loglevel             warning
logfile              ""
databases            1
always-show-logo     no
set-proc-title       yes
proc-title-template  "{title} paperless-dedicated instance"
# Working directory; database files go here
dir                  /srv/paperless/kvdb/
# Default database file
dbfilename           dump.rdb
# Persist the DB after 300 seconds (5 minutes) if least 1 key has changed
save                 300 1
## Write an additional append-only database file for increased crash resilience
#appendonly           yes
#appendfilename       appendonly.aof
#appendfsync          everysec

Q: What are the recommendations to avoid this bug?

A: To avoid this bug, we recommend the following:

  1. Carefully review the migration documentation for Valkey 8.1.0 to ensure that all necessary steps are taken to preserve the contents of the dump.rdb file.
  2. Verify that the Valkey configuration has the append-only-file (AOF) functionality commented out before the first start.
  3. Consider using a different migration strategy, such as a logical migration, to minimize the risk of data loss.
  4. Regularly back up the Redis database before attempting a migration to ensure that data can be recovered in case of an issue.

Q: What are the potential consequences of data loss during a migration?

A: The potential consequences of data loss during a migration can be severe, including:

  • Loss of critical business data
  • Downtime and revenue loss
  • Damage to reputation and customer trust
  • Increased costs for data recovery and restoration

Q: How can I prevent data loss during a migration?

A: To prevent data loss during a migration, follow these best practices:

  • Regularly back up the source database before attempting a migration
  • Use a reliable and tested migration strategy
  • Verify that the target database is properly configured and ready for data transfer
  • Monitor the migration process closely and address any issues promptly
  • Test the migrated database thoroughly to ensure data integrity and consistency