- 12 Jan, 2021 1 commit
-
- 27 Oct, 2020 12 commits
-
-
Oran Agra authored
In some cases one command added a very big bulk of memory, and this would be "resolved" by the eviction before the next command. Seeing an unexplained mass eviction we would wish to know the highest momentary usage too. Tracking it in call() and beforeSleep() adds some hooks in AOF and RDB loading. The fix in clientsCronTrackExpansiveClients is related to #7874 (cherry picked from commit 457b7073)
-
Yossi Gottlieb authored
Avoid using a static buffer for short key index responses, and make it caller's responsibility to stack-allocate a result type. Responses that don't fit are still allocated on the heap. (cherry picked from commit 9b7f8ba8)
-
Oran Agra authored
track and report memory used by clients argv. this is very usaful in case clients started sending a command and didn't complete it. in which case the first args of the command are already trimmed from the query buffer. in an effort to avoid cache misses and overheads while keeping track of these, i avoid calling sdsZmallocSize and instead use the sdslen / bulk-len which can at least give some insight into the problem. This memory is now added to the total clients memory usage, as well as the client list. (cherry picked from commit bea40e6a)
-
Wang Yuan authored
Before this commit, we would have continued to add replies to the reply buffer even if client output buffer limit is reached, so the used memory would keep increasing over the configured limit. What's more, we shouldn’t write any reply to the client if it is set 'CLIENT_CLOSE_ASAP' flag because that doesn't conform to its definition and we will close all clients flagged with 'CLIENT_CLOSE_ASAP' in ‘beforeSleep’. Because of code execution order, before this, we may firstly write to part of the replies to the socket before disconnecting it, but in fact, we may can’t send the full replies to clients since OS socket buffer is limited. But this unexpected behavior makes some commands work well, for instance ACL DELUSER, if the client deletes the current user, we need to send reply to client and close the connection, but before, we close the client firstly and write the reply to reply buffer. secondly, we shouldn't do this despite the fact it works well in most cases. We add a flag 'CLIENT_CLOSE_AFTER_COMMAND' to mark clients, this flag means we will close the client after executing commands and send all entire replies, so that we can write replies to reply buffer during executing commands, send replies to clients, and close them later. We also fix some implicit problems. If client output buffer limit is enforced in 'multi/exec', all commands will be executed completely in redis and clients will not read any reply instead of partial replies. Even more, if the client executes 'ACL deluser' the using user in 'multi/exec', it will not read the replies after 'ACL deluser' just like before executing 'client kill' itself in 'multi/exec'. We added some tests for output buffer limit breach during multi-exec and using a pipeline of many small commands rather than one with big response. Co-authored-by:
Oran Agra <oran@redislabs.com> (cherry picked from commit 57709c4b)
-
WuYunlong authored
Refine comment of makeThreadKillable(). This commit can be backported to 5.0, only if we also backport 8b70cb0e . Co-authored-by:
Oran Agra <oran@redislabs.com> (cherry picked from commit 647cac5b)
-
Wang Yuan authored
We're already using bg_unlink in several places to delete the rdb file in the background, and avoid paying the cost of the deletion from our main thread. This commit uses bg_unlink to remove the temporary rdb file in the background too. However, in case we delete that rdb file just before exiting, we don't actually wait for the background thread or the main thread to delete it, and just let the OS clean up after us. i.e. we open the file, unlink it and exit with the fd still open. Furthermore, rdbRemoveTempFile can be called from a thread and was using snprintf which is not async-signal-safe, we now use ll2string instead. (cherry picked from commit b002d2b4)
-
WuYunlong authored
If one thread got SIGSEGV, function sigsegvHandler() would be triggered, it would call bioKillThreads(). But call pthread_cancel() to cancel itself would make it block. Also note that if SIGSEGV is caught by bio thread, it should kill the main thread in order to give a positive report. (cherry picked from commit 8b70cb0e)
-
Oran Agra authored
List of squashed commits or PRs =============================== commit 66801ea Author: hwware <wen.hui.ware@gmail.com> Date: Mon Jan 13 00:54:31 2020 -0500 typo fix in acl.c commit 46f55db Author: Itamar Haber <itamar@redislabs.com> Date: Sun Sep 6 18:24:11 2020 +0300 Updates a couple of comments Specifically: * RM_AutoMemory completed instead of pointing to docs * Updated link to custom type doc commit 61a2aa0 Author: xindoo <xindoo@qq.com> Date: Tue Sep 1 19:24:59 2020 +0800 Correct errors in code comments commit a5871d1 Author: yz1509 <pro-756@qq.com> Date: Tue Sep 1 18:36:06 2020 +0800 fix typos in module.c commit 41eede7 Author: bookug <bookug@qq.com> Date: Sat Aug 15 01:11:33 2020 +0800 docs: fix typos in comments commit c303c84 Author: lazy-snail <ws.niu@outlook.com> Date: Fri Aug 7 11:15:44 2020 +0800 fix spelling in redis.conf commit 1eb76bf Author: zhujian <zhujianxyz@gmail.com>...
-
- 10 Sep, 2020 3 commits
-
-
Yossi Gottlieb authored
This is a catch-all test to confirm that that rewrite produces a valid output for all parameters and that this process does not introduce undesired configuration changes. (cherry picked from commit a8b72689)
-
- 01 Sep, 2020 10 commits
-
-
Yossi Gottlieb authored
Add Linux kernel OOM killer control option. This adds the ability to control the Linux OOM killer oom_score_adj parameter for all Redis processes, depending on the process role (i.e. master, replica, background child). A oom-score-adj global boolean flag control this feature. In addition, specific values can be configured using oom-score-adj-values if additional tuning is required. (cherry picked from commit 2530dc0e)
-
Arun Ranganathan authored
Co-authored-by:
Oran Agra <oran@redislabs.com> (cherry picked from commit f6cad30b)
-
Oran Agra authored
65a3307b (released in 6.0.6) has a side effect, when processCommand rejects a command with pre-made shared object error string, it trims the newlines from the end of the string. if that string is later used with addReply, the newline will be missing, breaking the protocol, and leaving the client hung. It seems that the only scenario which this happens is when replying with -LOADING to some command, and later using that reply from the CONFIG SET command (still during loading). this will result in hung client. Refactoring the code in order to avoid trimming these newlines from shared string objects, and do the newline trimming only in other cases where it's needed. Co-authored-by:
Guy Benoish <guy.benoish@redislabs.com> (cherry picked from commit 9fcd9e19)
-
valentinogeron authored
If the server gets MULTI command followed by only read commands, and right before it gets the EXEC it reaches OOM, the client will get OOM response. So, from now on, it will get OOM response only if there was at least one command that was tagged with `use-memory` flag (cherry picked from commit b7289e91)
-
杨博东 authored
After fork, the child process(redis-aof-rewrite) will get the fd opened by the parent process(redis), when redis killed by kill -9, it will not graceful exit(call prepareForShutdown()), so redis-aof-rewrite thread may still alive, the fd(lock) will still be held by redis-aof-rewrite thread, and redis restart will fail to get lock, means fail to start. This issue was causing failures in the cluster tests in github actions. Co-authored-by:
Oran Agra <oran@redislabs.com> (cherry picked from commit cbaf3c5b)
-
Yossi Gottlieb authored
Initialize and configure OpenSSL even when tls-port is not used, because we may still have tls-cluster or tls-replication. Also, make sure to reconfigure OpenSSL when these parameters are changed as TLS could have been enabled for the first time. (cherry picked from commit c75512d8)
-
Madelyn Olson authored
(cherry picked from commit 818dc3a0)
- 20 Jul, 2020 3 commits
-
-
dmurnane authored
Co-authored-by:
Daniel Murnane <dmurnane@eitccorp.com> (cherry picked from commit 9242ccf2)
-
Oran Agra authored
In order to support the use of multi-exec in pipeline, it is important that MULTI and EXEC are never rejected and it is easy for the client to know if the connection is still in multi state. It was easy to make sure MULTI and DISCARD never fail (done by previous commits) since these only change the client state and don't do any actual change in the server, but EXEC is a different story. Since in the past, it was possible for clients to handle some EXEC errors and retry the EXEC, we now can't affort to return any error on EXEC other than EXECABORT, which now carries with it the real reason for the abort too. Other fixes in this commit: - Some checks that where performed at the time of queuing need to be re- validated when EXEC runs, for instance if the transaction contains writes commands, it needs to be aborted. there was one check that was already done in execCommand (-READONLY), but other checks where missing: -OOM, -MISCONF, -NOREPLICAS, -MASTERDOWN - When a command is rejected by processCommand it was rejected with addReply, which was not recognized as an error in case the bad command came from the master. this will enable to count or MONITOR these errors in the future. - make it easier for tests to create additional (non deferred) clients. - add tests for the fixes of this commit. (cherry picked from commit 65a3307b)
-
Tomasz Poradowski authored
- enforcing of SHUTDOWN_NOSAVE flag in one place to make it consitent when running in Sentinel mode (cherry picked from commit 4ee011ad)
-
- 12 Jun, 2020 2 commits
-
-
antirez authored
-
Paul Spooren authored
The `LRANK` command returns the index (position) of a given element within a list. Using the `direction` argument it is possible to specify going from head to tail (acending, 1) or from tail to head (decending, -1). Only the first found index is returend. The complexity is O(N). When using lists as a queue it can be of interest at what position a given element is, for instance to monitor a job processing through a work queue. This came up within the Python `rq` project which is based on Redis[0]. [0]: https://github.com/rq/rq/issues/1197 Signed-off-by:
Paul Spooren <mail@aparcar.org>
-
- 09 Jun, 2020 1 commit
-
-
Oran Agra authored
Much like MULTI/EXEC/DISCARD, the WATCH and UNWATCH are not actually operating on the database or server state, but instead operate on the client state. the client may send them all in one long pipeline and check all the responses only at the end, so failing them may lead to a mismatch between the client state on the server and the one on the client end, and execute the wrong commands (ones that were meant to be discarded) the watched keys are not actually stored in the client struct, but they are in fact part of the client state. for instance, they're not cleared or moved in SWAPDB or FLUSHDB.
-
- 28 May, 2020 1 commit
-
-
antirez authored
After a closer look, the Redis core devleopers all believe that this was too fragile, caused many bugs that we didn't expect and that were very hard to track. Better to find an alternative solution that is simpler.
-
- 14 May, 2020 4 commits
-
-
antirez authored
Related to #7234.
-
antirez authored
-
Oran Agra authored
This bug was introduced by a recent change in which readQueryFromClient is using freeClientAsync, and despite the fact that now freeClientsInAsyncFreeQueue is in beforeSleep, that's not enough since it's not called during loading in processEventsWhileBlocked. furthermore, afterSleep was called in that case but beforeSleep wasn't. This bug also caused slowness sine the level-triggered mode of epoll kept signaling these connections as readable causing us to keep doing connRead again and again for ll of these, which keep accumulating. now both before and after sleep are called, but not all of their actions are performed during loading, some are only reserved for the main loop. fixes issue #7215
-
antirez authored
A fix for #7249.
-
- 08 May, 2020 3 commits