- 10 Jul, 2023 2 commits
-
-
sundb authored
This was introduced by the recent change in #11692 which prevented a down-sizing rehashing while there is a fork. ## Solution 1. Fix the rehashing code, so that the same as it allows rehashing for up-sizing during fork when the ratio is extreme, it will allow it for down-sizing as well. Co-authored-by:
Oran Agra <oran@redislabs.com> This is a partial cherry pick of: (cherry picked from commit b00a2351) (cherry picked from commit d4c37320382edb342292a3e30250d46896a12016)
-
Oran Agra authored
* Fix integer overflows due to using wrong integer size. * Add assertions / panic when overflow still happens. * Deletion of dead code to avoid need to maintain it * Some changes are not because of bugs, but rather paranoia. * Improve cmsgpack and cjson test coverage. Co-authored-by:
Yossi Gottlieb <yossigo@gmail.com>
-
- 17 Apr, 2023 6 commits
-
-
Oran Agra authored
-
Binbin authored
This bug seems to be there forever, CLIENT REPLY OFF|SKIP will mark the client with CLIENT_REPLY_OFF or CLIENT_REPLY_SKIP flags. With these flags, prepareClientToWrite called by addReply* will return C_ERR directly. So the client can't receive the Pub/Sub messages and any other push notifications, e.g client side tracking. In this PR, we adding a CLIENT_PUSHING flag, disables the reply silencing flags. When adding push replies, set the flag, after the reply, clear the flag. Then add the flag check in prepareClientToWrite. Fixes #11874 Note, the SUBSCRIBE command response is a bit awkward, see https://github.com/redis/redis-doc/pull/2327 Co-authored-by:
Oran Agra <oran@redislabs.com> (cherry picked from commit 416842e6) (cherry picked from commit f8ae7a41)
-
Madelyn Olson authored
This change attempts to alleviate a minor memory usage degradation for Redis 6.2 and onwards when using rather large objects (~2k) in streams. Introduced in #6281, we pre-allocate the head nodes of a stream to be 4kb, to limit the amount of unnecessary initial reallocations that are done. However, if we only ever allocate one object because 2 objects exceeds the max_stream_entry_size, we never actually shrink it to fit the single item. This can lead to a lot of excessive memory usage. For smaller item sizes this becomes less of an issue, as the overhead decreases as the items become smaller in size. This commit also changes the MEMORY USAGE of streams, since it was reporting the lpBytes instead of the allocated size. This introduced an observability issue when diagnosing the memory issue, since Redis reported the same amount of used bytes pre and post change, even though the new implementation allocated more memory. (cherry picked from commit 2bb29e4a) (cherry picked from commit 17181517)
-
Slava Koyfman authored
The existing logic for killing pub-sub clients did not handle the `allchannels` permission correctly. For example, if you: ACL SETUSER foo allchannels Have a client authenticate as the user `foo` and subscribe to a channel, and then: ACL SETUSER foo resetchannels The subscribed client would not be disconnected, though new clients under that user would be blocked from subscribing to any channels. This was caused by an incomplete optimization in `ACLKillPubsubClientsIfNeeded` checking whether the new channel permissions were a strict superset of the old ones. (cherry picked from commit f38aa6bf) (cherry picked from commit 9caeadb8661f5fabd8386e3f07d445fe6ca46e7f)
-
chendianqiang authored
Check the validity of the value before performing the create operation, prevents new data from being generated even if the request fails to execute. Co-authored-by:
Oran Agra <oran@redislabs.com> Co-authored-by:
chendianqiang <chendianqiang@meituan.com> Co-authored-by:
Binbin <binloveplay1314@qq.com> (cherry picked from commit bc7fe41e) (cherry picked from commit 606a385935363ea46c0df4f40f8a949d85f7a20a)
-
- 28 Feb, 2023 8 commits
-
-
Oran Agra authored
-
Oran Agra authored
Issue happens when passing a negative long value that greater than the max positive value that the long can store. (cherry picked from commit 41430af6a821c551abb862666ef896f2c196dea6)
-
Tom Levy authored
Authenticated users can use string matching commands with a specially crafted pattern to trigger a denial-of-service attack on Redis, causing it to hang and consume 100% CPU time. (cherry picked from commit e75f92047c22e659d49bba3a083cd0c9935f21e6)
-
Madelyn Olson authored
-
Madelyn Olson authored
(cherry picked from commit f7150c45bc5d6f03c8ba86a9a9296d024c6848dc)
-
zhaozhao.zz authored
(cherry picked from commit a35e0837) (cherry picked from commit 76473f50990e06872d5a08886549815077f5def5)
-
uriyage authored
In #7875 (Redis 6.2), we changed the sds alloc to be the usable allocation size in order to: > reduce the need for realloc calls by making the sds implicitly take over the internal fragmentation This change was done most sds functions, excluding `sdsRemoveFreeSpace` and `sdsResize`, the reason is that in some places (e.g. clientsCronResizeQueryBuffer) we call sdsRemoveFreeSpace when we see excessive free space and want to trim it. so if we don't trim it exactly to size, the caller may still see excessive free space and call it again and again. However, this resulted in some excessive calls to realloc, even when there's no need and it's gonna be a no-op (e.g. when reducing 15 bytes allocation to 13). It turns out that a call for realloc with jemalloc can be expensive even if it ends up doing nothing, so this PR adds a check using `je_nallocx`, which is cheap to avoid the call for realloc. in addition to that this PR unifies sdsResize and sdsRemoveFreeSpace into common code. the difference between them was that sdsResize would avoid using SDS_TYPE_5, since it want to keep the string ready to be resized again, while sdsRemoveFreeSpace would permit using SDS_TYPE_5 and get an optimal memory consumption. now both methods take a `would_regrow` argument that makes it more explicit. the only actual impact of that is that in clientsCronResizeQueryBuffer we call both sdsResize and sdsRemoveFreeSpace for in different cases, and we now prevent the use of SDS_TYPE_5 in both. The new test that was added to cover this concern used to pass before this PR as well, this PR is just a performance optimization and cleanup. Benchmark: `redis-benchmark -c 100 -t set -d 512 -P 10 -n 100000000` on i7-9850H with jemalloc, shows improvement from 1021k ops/sec to 1067k (average of 3 runs). some 4.5% improvement. Co-authored-by:
Oran Agra <oran@redislabs.com> (cherry picked from commit 46393f98) (cherry picked from commit b12eeccddd9318a5d97a5aee2dad88999dfad53f)
-
Oran Agra authored
Turns out that a fork child calling getExpire while persisting keys (and possibly also a result of some module fork tasks) could cause dictFind to do incremental rehashing in the child process, which is both a waste of time, and also causes COW harm. (cherry picked from commit 2bec254d) (cherry picked from commit 3e82bdf7)
-
- 17 Jan, 2023 2 commits
-
-
Oran Agra authored
-
Oran Agra authored
in Redis 7.0 this fix covers KEYS as well, but in 6.2 and 6.0 it doesn't, this is because in 7.0 there's a mechanism to avoid sending partial replies to the client, and in older releases there isn't, and without it there's a risk that the client would be able to read what looks like a complete KEYS command.
-
- 16 Jan, 2023 5 commits
-
-
Oran Agra authored
-
Oran Agra authored
missing range check in ZRANDMEMBER and HRANDIFLD leading to panic due to protocol limitations
-
Oran Agra authored
Authenticated users issuing specially crafted SETRANGE and SORT(_RO) commands can trigger an integer overflow, resulting with Redis attempting to allocate impossible amounts of memory and abort with an OOM panic.
-
Oran Agra authored
Related to the hang reported in #11671 Currently, redis can disconnect a client due to reaching output buffer limit, it'll also avoid feeding that output buffer with more data, but it will keep running the loop in the command (despite the client already being marked for disconnection) This PR is an attempt to mitigate the problem, specifically for commands that are easy to abuse, specifically: KEYS, HRANDFIELD, SRANDMEMBER, ZRANDMEMBER. The RAND family of commands can take a negative COUNT argument (which is not bound to the number of elements in the key), so it's enough to create a key with one field, and then these commands can be used to hang redis. For KEYS the caller can use the existing keyspace in redis (if big enough).
-
Moti Cohen authored
As Sentinel supports dynamic IP only when using hostnames, there are few leftover addess comparison logic that doesn't take into account that the IP might get change. Co-authored-by:
moticless <moticless@github.com> (cherry picked from commit 4a27aa48)
-
- 12 Dec, 2022 17 commits
-
-
Oran Agra authored
-
Yossi Gottlieb authored
Before this commit, TLS tests on Ubuntu 22.04 would fail as dropped connections result with an ECONNABORTED error thrown instead of an empty read. (cherry picked from commit 69d55768)
-
Yossi Gottlieb authored
This solves several problems in a more elegant way: * No need to explicitly use `-lc` on x86_64 when building with `-m32`. * Avoids issues with undefined floating point emulation funcs on ARM. (cherry picked from commit f26e90be)
-
Yossi Gottlieb authored
* Fix test modules linking on macOS 11.x. * Use macOS 10.x for FreeBSD VM as VirtualBox is not yet supported on 11. (cherry picked from commit 6d5a9117)
-
Ozan Tezcan authored
**Signed integer overflow.** Although, signed overflow issue can be problematic time to time and change how compiler generates code, current findings mostly about signed shift or simple addition overflow. For most platforms Redis can be compiled for, this wouldn't cause any issue as far as I can tell (checked generated code on godbolt.org). UB means nothing guaranteed and risky to reason about program behavior but I don't think any of the fixes here worth backporting. As sanitizers are now part of the CI, preventing new issues will be the real benefit. partial cherry pick from commit b91d8b28 The bug in BITFIELD seems to affect 12.2.1 used on Alpine
-
Madelyn Olson authored
(cherry picked from commit 362b3b02)
-
Oran Agra authored
-
Oran Agra authored
-
Oran Agra authored
This test sets the master ping interval to 1 hour, in order to avoid pings in the replicatoin stream incrementing the replication offset, however, it didn't increase the repl-timeout so on slow machines where the test took more than 60 seconds, the replicas would drop and reconnect. ``` *** [err]: PSYNC2: Partial resync after restart using RDB aux fields in tests/integration/psync2.tcl Replica didn't partial sync ``` The test would detect 4 additional partial syncs where it expects only one. (cherry picked from commit b0250b45)
-
chenyang8094 authored
The warning: ``` pqsort.c:106:7: warning: performing pointer subtraction with a null pointer has undefined behavior [-Wnull-pointer-subtraction] loop: SWAPINIT(a, es); ^~~~~~~~~~~~~~~ pqsort.c:65:47: note: expanded from macro 'SWAPINIT' #define SWAPINIT(a, es) swaptype = ((char *)a - (char *)NULL) % sizeof(long) || \ ``` Clang version: ``` Apple clang version 13.1.6 (clang-1316.0.21.2) Target: x86_64-apple-darwin21.3.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin ``` (cherry picked from commit cb625844)
-
uriyage authored
In moduleFireServerEvent we change the real client DB to 0 on freeClient in case the event is REDISMODULE_EVENT_CLIENT_CHANGE. It results in a crash if the client is blocked on a key on other than DB 0. The DB change is not necessary even for module-client, as we set its DB to 0 on either createClient or moduleReleaseTempClient. Co-authored-by:
Madelyn Olson <34459052+madolson@users.noreply.github.com> Co-authored-by:
Binbin <binloveplay1314@qq.com> Co-authored-by:
Oran Agra <oran@redislabs.com> (cherry picked from commit e4eb18b3)
-
Oran Agra authored
Fix a few issues with the recent #11463 * use exitFromChild instead of exit * test should ignore defunct process since that's what we expect to happen for thees child processes when the parent dies. * fix typo Co-authored-by:
Binbin <binloveplay1314@qq.com> (cherry picked from commit 4c54528f)
-
Oran Agra authored
During a diskless sync, if the master main process crashes, the child would have hung in `write`. This fix closes the read fd on the child side, so that if the parent crashes, the child will get a write error and exit. This change also fixes disk-based replication, BGSAVE and AOFRW. In that case the child wouldn't have been hang, it would have just kept running until done which may be pointless. There is a certain degree of risk here. in case there's a BGSAVE child that could maybe succeed and the parent dies for some reason, the old code would have let the child keep running and maybe succeed and avoid data loss. On the other hand, if the parent is restarted, it would have loaded an old rdb file (or none), and then the child could reach the end and rename the rdb file (data conflicting with what the parent has), or also have a race with another BGSAVE child that the new parent started. Note that i removed a comment saying a write error will be ignored in the child and handled by the parent (this comment was very old and i don't think relevant). (cherry picked from commit ccaef5c9)
-
Moti Cohen authored
Funcion sentinelAddrEqualsHostname() of sentinel makes DNS resolve and based on it determines if two IP addresses are equal. Now, If the DNS resolve command fails, the function simply returns 0, even if the hostnames are identical. This might become an issue in case of failover such that sentinel might receives from Redis instance, response to regular INFO query it sent, and wrongly decide that the instance is pointing to is different leader than the one recorded because of this function, yet hostnames are identical. In turn sentinel disconnects the connection between sentinel and valid slave which leads to -failover-abort-no-good-slave. See issue #11241. I managed to reproduce only part of the flow in which the function return wrong result and trigger +fix-slave-config. The fix is even if the function failed to resolve then compare based on hostnames. That is our best effort as long as the server is unavailable for some reason. It is fine since Redis instance cannot have multiple hostnames for a given setup (cherry picked from commit bd23b15a)
-
Meir Shpilraien (Spielrein) authored
If Redis crashes due to calling an invalid function pointer, the `backtrace` function will try to dereference this invalid pointer which will cause a crash inside the crash report and will kill the processes without having all the crash report information. Example: ``` === REDIS BUG REPORT START: Cut & paste starting from here === 198672:M 19 Sep 2022 18:06:12.936 # Redis 255.255.255 crashed by signal: 11, si_code: 1 198672:M 19 Sep 2022 18:06:12.936 # Accessing address: 0x1 198672:M 19 Sep 2022 18:06:12.936 # Crashed running the instruction at: 0x1 // here the processes is crashing ``` This PR tries to fix this crash be: 1. Identify the issue when it happened. 2. Replace the invalid pointer with a pointer to some dummy function so that `backtrace` will not crash. I identification is done by comparing `eip` to `info->si_addr`, if they are the same we know that the crash happened on the same address it tries to accesses and we can conclude that it tries to call and invalid function pointer. To replace the invalid pointer we introduce a new function, `setMcontextEip`, which is very similar to `getMcontextEip` and it knows to set the Eip for the different supported OS's. After printing the trace we retrieve the old `Eip` value. (cherry picked from commit 0bf90d94)
-
zhaozhao.zz authored
This bug is introduced in #7653. (Redis 6.2.0) When `server.maxmemory_eviction_tenacity` is 100, `eviction_time_limit_us` is `ULONG_MAX`, and if we cannot find the best key to delete (e.g. maxmemory-policy is `volatile-lru` and all keys with ttl have been evicted), in `cant_free` redis will sleep forever if some items are being freed in the lazyfree thread. (cherry picked from commit 464aa041)
-