- 12 Jan, 2021 20 commits
-
-
huangzhw authored
This is a recent problem, introduced by 74717438 (redis 6.0) The implications are: The sole difference between LookupKeyRead and LookupKeyWrite is for command executed on a replica, which are not received from its master client. (for the master, and for the master client on the replica, these two functions behave the same)! Since SORT is a write command, this bug only implicates a writable-replica. And these are its implications: - SORT STORE will behave as it did before the above mentioned commit (like before redis 6.0). on a writable-replica an already logically expired the key would have appeared missing. (store dest key would be deleted, instead of being populated with the data from the already logically expired key) - SORT (the non store variant, which in theory could have been executed on read-only-replica if it weren't for the write flag), will (in redis 6.0) have a new bug and return the data from the already logically expired key instead of empty response. (cherry picked from commit 08ad6abd)
-
Oran Agra authored
Turns out the RDB checksum in Redis 6.0 on bigendian is broken. It always returned 0, so the RDB files are generated as if checksum is disabled, and will be loaded ok on littleendian, and on bigendian. But it'll not be able to load RDB files generated on littleendian or older versions. Similarly DUMP and RESTORE will work on the same version (0==0), but will be unable to exchange dump payloads with littleendian or old versions. (cherry picked from commit 324070c8)
-
Oran Agra authored
getRDB is "designed" to work in two modes: one for redis-cli --rdb and one for redis-cli --cluster backup. in the later case it uses the hiredis connection from the cluster nodes and it used to free it without nullifying the context, so a later attempt to free the context would crash. I suppose the reason it seems to want to free the hiredis context ASAP is that it wants to disconnect the replica link, so that replication buffers will not be accumulated. (cherry picked from commit 41b2ed2b)
-
Oran Agra authored
When a Lua script returns a map to redis (a feature which was added in redis 6 together with RESP3), it would have returned the value first and the key second. If the client was using RESP2, it was getting them out of order, and if the client was in RESP3, it was getting a map of value => key. This was happening regardless of the Lua script using redis.setresp(3) or not. This also affects a case where the script was returning a map which it got from from redis by doing something like: redis.setresp(3); return redis.call() This fix is a breaking change for redis 6.0 users who happened to rely on the wrong order (either ones that used redis.setresp(3), or ones that returned a map explicitly). This commit also includes other two changes in the tests: 1. The test suite now handles RESP3 maps as dicts rather than nested lists 2. Remove some redundant (duplicate) tests from tracking.tcl (cherry picked from commit 2017407b)
-
Oran Agra authored
The crash log attempts to print the current client info, and when it does that it attempts to check if the first argument happens to be a key but it did so for commands with no arguments too, which caused the crash log to crash half way and not reach its end. (cherry picked from commit 152b5d46)
-
Yossi Gottlieb authored
Normally IO threads should simply read data from the socket into the buffer and attempt to parse it. If a protocol error is detected, a reply is generated which may result with installing a write handler which is not thread safe. This fix delays that until the client is processed back in the main thread. Fixes #8220 (cherry picked from commit e7047ec2)
-
George Prekas authored
Older arm64 Linux kernels have a bug that could lead to data corruption during background save under the following scenario: 1) jemalloc uses MADV_FREE on a page, 2) jemalloc reuses and writes the page, 3) Redis forks the background save process, and 4) Linux performs page reclamation. Under these conditions, Linux will reclaim the page wrongfully and the background save process will read zeros when it tries to read the page. The bug has been fixed in Linux with commit: ff1712f953e27f0b0718762ec17d0adb15c9fd0b ("arm64: pgtable: Ensure dirty bit is preserved across pte_wrprotect()") This Commit adds an ignore-warnings config, when not found, redis will print a warning and exit on startup (default behavior). Co-authored-by:
Oran Agra <oran@redislabs.com> (cherry picked from commit b02780c4) in 6.0 this warning is ignored by default in order to avoid adding regression, specifically for deployments that don't need persistence or replication
-
Qu Chen authored
In response to large client query buffer optimization introduced in 1898e6ce. The calculation of the amount of remaining bytes we need to write to the query buffer was calculated wrong, as a result we are unnecessarily growing the client query buffer by sdslen(c->querybuf) always. This fix corrects that behavior. Please note the previous behavior prior to the before-mentioned change was correctly calculating the remaining additional bytes, and this change makes that calculate to be consistent. Useful context, the argument of size `ll` starts at qb_pos (which is now the beginning of the sds), but much of it may have already been read from the socket, so we only need to grow the sds for the remainder of it. (cherry picked from commit 11b3325e)
-
Oran Agra authored
Module blocked clients cache the response in a temporary client, the reply list in this client would be affected by the recent fix in #7202, but when the reply is later copied into the real client, it would have bypassed all the checks for output buffer limit, which would have resulted in both: responding with a partial response to the client, and also not disconnecting it at all. (cherry picked from commit 48efc25f)
-
Yossi Gottlieb authored
Makes spt_init more careful with assumptions about what memory regions may be overwritten. It will now only consider a contiguous block of argv and envp elements and mind any gaps. (cherry picked from commit ec02c761)
-
Yossi Gottlieb authored
Seems to have gone unnoticed for a long time, because at least with glibc it will only be triggered if setenv() was called before spt_init, which Redis doesn't. Fixes #8064. (cherry picked from commit 7e5a6313)
-
Wang Yuan authored
Backup keys to slots map and restore when fail to sync if diskless-load type is swapdb in cluster mode (#8108) When replica diskless-load type is swapdb in cluster mode, we didn't backup keys to slots map, so we will lose keys to slots map if fail to sync. Now we backup keys to slots map at first, and restore it properly when fail. This commit includes a refactory/cleanup of the backups mechanism (moving it to db.c and re-structuring it a bit). Co-authored-by:
Oran Agra <oran@redislabs.com> (cherry picked from commit b55a827e)
-
Oran Agra authored
This is hopefully usually harmles. The server.ready_keys will usually be empty so the code after releasing the GIL will soon be done. The only case where it'll actually process things is when a module releases a client (or module) blocked on a key, by triggering this NOT from within a command (e.g. a timer event). This bug was introduced in redis 6.0.9, see #7903 (cherry picked from commit e6fa4738)
-
Oran Agra authored
Fix: When oom-score-adj-values is provided in the config file after oom-score-adj yes, it'll take an immediate action, before readOOMScoreAdj was acquired, resulting in an error (out of range score due to uninitialized value. delay the reaction the real call is made by main(). Since the values are clamped to -1000..1000, and they're applied as an offset from the value at startup (which may be -1000), we need to allow the offsets to reach to +2000 so that a value of +1000 is achievable in case the value at startup was -1000. Adding an option for absolute values rather than relative ones. (cherry picked from commit 61954951)
-
guybe7 authored
The bug was introduced by #5021 which only attempted avoid EXIST on an already expired key from returning 1 on a replica. Before that commit, dbExists was used instead of lookupKeyRead (which had an undesired effect to "touch" the LRU/LFU) Other than that, this commit fixes OBJECT to also come empty handed on expired keys in replica. And DEBUG DIGEST-VALUE to behave like DEBUG OBJECT (get the data from the key regardless of it's expired state) (cherry picked from commit f8ae9917)
- 27 Oct, 2020 20 commits
-
-
Oran Agra authored
-
Yossi Gottlieb authored
When using a system with no malloc_usable_size(), zmalloc_size() assumed that the heap allocator always returns blocks that are long-padded. This may not always be the case, and will result with zmalloc_size() returning a size that is bigger than allocated. At least in one case this leads to out of bound write, process crash and a potential security vulnerability. Effectively this does not affect the vast majority of users, who use jemalloc or glibc. This problem along with a (different) fix was reported by Drew DeVault. (cherry picked from commit 9824fe3e)
-
Oran Agra authored
The tests sometimes fail to find a log message. Recently i added a print that shows the log files that are searched and it shows that the message was in deed there. The only reason i can't think of for this seach to fail, is we we happened to read an incomplete line, which didn't match our pattern and then on the next iteration we would continue reading from the line after it. The fix is to always re-evaluation the previous line. (cherry picked from commit 4e2e5be2)
-
David CARLIER authored
(cherry picked from commit 27f4c212)
-
Zach Fewtrell authored
(cherry picked from commit ebfa7699)
-
Wen Hui authored
Previous code would have added default redis save parameters to the config file on rewrite, which would have been silently ignored when the config file is loaded. The new code avoids adding this, and also actively removes these lines If added by a previous config rewrite. (cherry picked from commit 0f370f9b)
-
Qu Chen authored
This wrong behavior was backed by a test, and also documentation, and dates back to 2010. But it makes no sense to anyone involved so it was decided to change that. Note that 20eeddfb (invalidate watch on expire on access) was released in 6.0 RC2 and 2d1968f8 released in in 6.0.0 GA (invalidate watch when key is evicted). both of which do similar changes. (cherry picked from commit 556acefe)
-
Yossi Gottlieb authored
Useful for running tests on systems which may be way slower than usual. (cherry picked from commit 843a13e8)
-
filipe oliveira authored
Fixed bug concerning redis-benchmark non clustered benchmark forcing always the same hash tag {tag} (#7931) Adding the ":{tag}" only if --cluster is used, so that when used against a proxy it generates traffic to all shards. Co-authored-by:
Oran Agra <oran@redislabs.com> (cherry picked from commit 6cf23d66)
-
Oran Agra authored
In some cases one command added a very big bulk of memory, and this would be "resolved" by the eviction before the next command. Seeing an unexplained mass eviction we would wish to know the highest momentary usage too. Tracking it in call() and beforeSleep() adds some hooks in AOF and RDB loading. The fix in clientsCronTrackExpansiveClients is related to #7874 (cherry picked from commit 457b7073)
-
Yossi Gottlieb authored
(cherry picked from commit ef92f507)
-
Oran Agra authored
This is a compatibility issue with redis 5.0 that was introduced by ACL. Before this commit, setting requirepass to an empty string will result in a server that needs an empty AUTH, unlike redis 5.0 which would accept connections without an AUTH. (cherry picked from commit 19418b6b)
-
Wang Yuan authored
If we fail or stop to rewrite aof, we need to remove temporary aof. We also remove temporary rdb when replicas abort to receive rdb. But currently we delete them in main thread, to avoid blocking, we should use bg_unlink to remove them in a background thread. Btw, we have already used this way to removed child process temporary rdb. (cherry picked from commit aaacb8c9)
-
Andreas Lind authored
(cherry picked from commit 8b497881)
-
Yossi Gottlieb authored
This cleans up and simplifies the API by passing the command name as the first argument. Previously the command name was specified explicitly, but was still included in the argv. (cherry picked from commit 056a43e1)
-