- 21 Jul, 2021 1 commit
-
-
Oran Agra authored
Starting redis 6.0 and the changes we made to the diskless master to be suitable for TLS, I made the master avoid reaping (wait3) the pid of the child until we know all replicas are done reading their rdb. I did that in order to avoid a state where the rdb_child_pid is -1 but we don't yet want to start another fork (still busy serving that data to replicas). It turns out that the solution used so far was problematic in case the fork child was being killed (e.g. by the kernel OOM killer), in that case there's a chance that we currently disabled the read event on the rdb pipe, since we're waiting for a replica to become writable again. and in that scenario the master would have never realized the child exited, and the replica will remain hung too. Note that there's no mechanism to detect a hung replica while it's in rdb transfer state. The solution here is to add another pipe which is used by the parent to tell the child it is safe to exit. this mean that when the child exits, for whatever reason, it is safe to reap it. Besides that, i'm re-introducing an adjustment to REPLCONF ACK which was part of #6271 (Accelerate diskless master connections) but was dropped when that PR was rebased after the TLS fork/pipe changes (5a477946). Now that RdbPipeCleanup no longer calls checkChildrenDone, and the ACK has chance to detect that the child exited, it should be the one to call it so that we don't have to wait for cron (server.hz) to do that. (cherry picked from commit 573246f7)
-
- 01 Jun, 2021 5 commits
-
-
M Sazzadul Hoque authored
-
Oran Agra authored
-
patpatbear authored
this patch fixes sinterstore by add missing keyspace del event when any source set not exists. Co-authored-by:
srzhao <srzhao@sysnew.com> (cherry picked from commit 46d9f31e)
-
- 30 May, 2021 1 commit
-
-
ikeberlein authored
-
- 03 May, 2021 9 commits
-
-
Oran Agra authored
-
Oran Agra authored
An integer overflow bug in Redis 6.2 could be exploited to corrupt the heap and potentially result with remote code execution. The vulnerability involves changing the default set-max-intset-entries configuration value, creating a large set key that consists of integer values and using the COPY command to duplicate it. The integer overflow bug exists in all versions of Redis starting with 2.6, where it could result with a corrupted RDB or DUMP payload, but not exploited through COPY (which did not exist before 6.2). (cherry picked from commit 29900d4e)
-
Madelyn Olson authored
(cherry picked from commit 98d2e001)
-
Oran Agra authored
We sometimes see the crash report saying we were killed by a random process even in cases where the crash was spontanius in redis. for instance, crashes found by the corrupt-dump test. It looks like this si_pid is sometimes left uninitialized, and a good way to tell if the crash originated in redis or trigged by outside is to look at si_code, real signal codes are always > 0, and ones generated by kill are have si_code of 0 or below. (cherry picked from commit b45b0d81)
-
Yossi Gottlieb authored
listenToPort attempts to gracefully handle and ignore certain errors but does not store errno prior to logging, which in turn calls several libc functions that may overwrite errno. This has been discovered due to libmusl strftime() always returning with errno set to EINVAL, which resulted with docker-library/redis#273. (cherry picked from commit df5f543b)
-
guybe7 authored
Scenario: 1. A module client is blocked on keys with a timeout 2. Shortly before the timeout expires, the key is being populated and signaled as ready 3. Redis calls moduleTryServeClientBlockedOnKey (which replies to client) and then moduleUnblockClient 4. moduleUnblockClient doesn't really unblock the client, it writes to server.module_blocked_pipe and only marks the BC as unblocked. 5. beforeSleep kics in, by this time the client still exists and techincally timed-out. beforeSleep replies to the timeout client (double reply) and only then moduleHandleBlockedClients is called, reading from module_blocked_pipe and calling unblockClient The solution is similar to what was done in moduleTryServeClientBlockedOnKey: we should avoid re-processing an already-unblocked client (cherry picked from commit e58118cd)
-
- 02 Mar, 2021 4 commits
-
-
Oran Agra authored
-
Yossi Gottlieb authored
* Remove linux/version.h dependency. This introduces unnecessary dependencies, and generally not a good idea as the platform we build on may be different than the platform we run on. To determine if sync_file_range exists we can simply rely on header file hints. * Fix setproctitle() on libmusl. The previous ifdef checks were a bit too strict for no apparent reason. * Fix tests failure on Linux with no backtrace. * Add alpine daily CI job. (cherry picked from commit 95ea7454)
-
Yossi Gottlieb authored
Also adds a new daily CI test, relying on the fact that we don't use malloc_size() on alpine libmusl. Fixes #8531 (cherry picked from commit dd885780)
-
- 22 Feb, 2021 12 commits
-
-
Oran Agra authored
-
Yossi Gottlieb authored
On 32-bit systems, setting the proto-max-bulk-len config parameter to a high value may result with integer overflow and a subsequent heap overflow when parsing an input bulk (CVE-2021-21309). This fix has two parts: Set a reasonable limit to the config parameter. Add additional checks to prevent the problem in other potential but unknown code paths. (cherry picked from commit d32f2e99)
-
Viktor Söderqvist authored
Without this fix, RM_ZsetRem can leave empty sorted sets which are not allowed to exist. Removing from a sorted set while iterating seems to work (while inserting causes failed assetions). RM_ZsetRangeEndReached is modified to return 1 if the key doesn't exist, to terminate iteration when the last element has been removed. (cherry picked from commit aea6e71e)
-
Greg Femec authored
When a database on a 64 bit build grows past 2^31 keys, the underlying hash table expands to 2^32 buckets. After this point, the algorithms for selecting random elements only return elements from half of the available buckets because they use random() which has a range of 0 to 2^31 - 1. This causes problems for eviction policies which use dictGetSomeKeys or dictGetRandomKey. Over time they cause the hash table to become unbalanced because, while new keys are spread out evenly across all buckets, evictions come from only half of the available buckets. Eventually this half of the table starts to run out of keys and it takes longer and longer to find candidates for eviction. This continues until no more evictions can happen. This solution addresses this by using a 64 bit PRNG instead of libc random(). Co-authored-by:
Greg Femec <gfemec@google.com> (cherry picked from commit 266949c7)
-
filipe oliveira authored
Fix the pointers to the slot hash tags in the case of prefixed commands usage i.e. AUTH / SELECT It adjusts the pointers to the slot hash tags in the case of prefixed commands usage as soon as we get the 1st reply (same like we already did for the random strings within the command ) (cherry picked from commit efccd635)
-
zhaozhao.zz authored
This bug would have let users observe logically expired keys on replicas and during CLIENT PAUSE WRITE. (cherry picked from commit f2a5fe36)
-
Yossi Gottlieb authored
At least in one case the arm64 cow kernel bug test triggers an assert, which is a problem because it cannot be ignored like cases where the bug is found. On older systems (Linux <4.5) madvise fails because MADV_FREE is not supported. We treat these failures as an indication the system is not affected. Fixes #8351, #8406 (cherry picked from commit 3a504904)
-
Yossi Gottlieb authored
Fixes a regression introduced due to a new (safer) way of rewriting configuration files. In the past the file was simply overwritten (same inode), but now Redis creates a new temporary file and later renames it over the old one. The temp file typically gets created with 0600 permissions so we later fchmod it to fix that. Unlike open with O_CREAT, fchmod doesn't consider umask so we have to do that explicitly. Fixes #8369 (cherry picked from commit b548ffab)
-
guybe7 authored
The output for COMMAND command was wrong for some commands. clients can use firstkey,lastkey,step to find (some) key name arguments, and the "movablekeys" flag to know that they can't know all (or any) of the key name arguments. These commands had the wrong output: 1. GEORADIUS*_RO used to have "movablekeys" (which it doesn't really need) 2. XREAD and XREADGROUP used to have (1,1,1). but that's completely wrong. 3. Z*STORE used to have (0,0,0) but it can at lest give the index of the dstkey (1,1,1) (cherry picked from commit baf92f3f)
-
Madelyn Olson authored
clientsArePaused isn't thread safe because it has a side effect of attempting to unpause, which may cause multiple threads concurrently updating the unblocked_clients global list. This change resolves this issue by no longer postponing client for threaded reads when clients are paused and then skipping the check for client paused for threaded reads, in case one is postponed and then clients are paused. (I don't think this is strictly possible, but being defensive seems better here)
-
- 12 Jan, 2021 8 commits
-
-
Oran Agra authored
-
Oran Agra authored
c4fdf09c added a test that now fails with valgrind it fails for two resons: 1) the test samples the used memory and then limits the maxmemory to that value, but it turns out this is not atomic and on slow machines the background cron process that clean out old query buffers reduces the memory so that the setting doesn't cause eviction. 2) the dbsize was tested late, after reading some invalidation messages by that time more and more keys got evicted, partially draining the db. this is not the focus of this fix (still a known limitation) (cherry picked from commit a102b21d)
-
Oran Agra authored
The test was trying to wait for the replica to start loading the rdb from the master before it kills the master, but it was actually waiting for ROLE to be in "sync" mode, which corresponds to REPL_STATE_TRANSFER that starts before the actual loading starts. now instead it waits for the loading flag to be set. Besides, the test was dependent on the previous configuration of the servers, relying on the fact the replica is configured to persist (either RDB of AOF), now it is set explicitly. (cherry picked from commit 26495387)
-
Egor Seredin authored
This will allow to use: RedisModule_CreateStringPrintf(ctx, "%s %c %s", "string1", 0, "string2"); On large string, the previous code would incrementally retry to double the output buffer. now it uses the the return value of snprintf and grows to the right size in one step. and also avoids an excessive strlen in sdscat at the end. (cherry picked from commit f4ca3d87)
-
guybe7 authored
The bug occurs when 'callback' re-registers itself to a point in the future and the execution time in non-negligible: 'now' refers to time BEFORE callback was executed and is used to calculate 'next_period'. We must get the actual current time when calculating 'next_period' (cherry picked from commit 1a91a270)
-