- 09 Sep, 2020 6 commits
-
-
杨博东 authored
improves test coverage
-
Roi Lipman authored
Co-authored-by:
Yossi Gottlieb <yossigo@gmail.com> Co-authored-by:
Oran Agra <oran@redislabs.com>
-
Yossi Gottlieb authored
This is a catch-all test to confirm that that rewrite produces a valid output for all parameters and that this process does not introduce undesired configuration changes.
-
Yossi Gottlieb authored
Save parameters should either be default or whatever specified in the config file. This fixes an issue introduced in #7092 which causes configuration file settings to be applied on top of the defaults.
-
Yossi Gottlieb authored
-
Eran Liberty authored
There was a bug. Although cluster replicas would allow read commands, they would not allow a MULTI-EXEC that's composed solely of read commands. Adds tests for coverage. Co-authored-by:
Oran Agra <oran@redislabs.com> Co-authored-by:
Eran Liberty <eranl@amazon.com>
-
- 08 Sep, 2020 2 commits
-
-
Oran Agra authored
if there are nested tests and nested servers, we need to restore the previous value of cur_test when a test exist. example: ``` test{test 1} { start_server { test{test 1.1 - master only} { } start_server { test{test 1.2 - with replication} { } } } } ``` when `test 1.1 - master only exists`, we're still inside `test 1`
-
bodong.ybd authored
1) cur_test: when restart_server, "no such variable" error occurs ./runtest --single integration/rdb test {client freed during loading} SET ::cur_test restart_server kill_server test "Check for memory leaks (pid $pid)" SET ::cur_test UNSET ::cur_test UNSET ::cur_test // This global variable has been unset. 2) `ps --ppid` not available on macOS platform, can be replaced with `pgrep -P pid`.
-
- 07 Sep, 2020 3 commits
-
-
Oran Agra authored
This test was failing from time to time see discussion at the bottom of #7635 This was probably due to timing, the DEBUG SLEEP executed by redis-cli didn't sleep for enough time. This commit changes: 1) use SET-ACTIVE-EXPIRE instead of DEBUG SLEEP 2) reduce many `after` sleeps with retry loops to speed up the test. 3) add many comment explaining the different steps of the test and it's purpose. 4) config appendonly before populating the volatile keys, so that they'll be part of the AOF command stream rather than the preamble RDB portion. other complications: recently kill_instance switched from SIGKILL to SIGTERM, and this would sometimes fail since there was an AOFRW running in the background. now we wait for it to end before attempting the kill.
-
Yossi Gottlieb authored
There is an inherent race condition in port allocation for spawned servers. If a server fails to start because a port is taken, a new port is allocated. This fixes a problem where the logs are not truncated and as a result a large number of unmonitored servers are started.
-
- 06 Sep, 2020 8 commits
-
-
Oran Agra authored
Starting redis 6.0 and the changes we made to the diskless master to be suitable for TLS, I made the master avoid reaping (wait3) the pid of the child until we know all replicas are done reading their rdb. I did that in order to avoid a state where the rdb_child_pid is -1 but we don't yet want to start another fork (still busy serving that data to replicas). It turns out that the solution used so far was problematic in case the fork child was being killed (e.g. by the kernel OOM killer), in that case there's a chance that we currently disabled the read event on the rdb pipe, since we're waiting for a replica to become writable again. and in that scenario the master would have never realized the child exited, and the replica will remain hung too. Note that there's no mechanism to detect a hung replica while it's in rdb transfer state. The solution here is to add another pipe which is used by the parent to tell the child it is safe to exit. this mean that when the child exits, for whatever reason, it is safe to reap it. Besides that, i'm re-introducing an adjustment to REPLCONF ACK which was part of #6271 (Accelerate diskless master connections) but was dropped when that PR was rebased after the TLS fork/pipe changes (5a477946). Now that RdbPipeCleanup no longer calls checkChildrenDone, and the ACK has chance to detect that the child exited, it should be the one to call it so that we don't have to wait for cron (server.hz) to do that.
-
Oran Agra authored
- redirect valgrind reports to a dedicated file rather than console - try to avoid killing instances with SIGKILL so that we get the memory leak report (killing with SIGTERM before resorting to SIGKILL) - search for valgrind reports when done, print them and fail the tests - add --dont-clean option to keep the logs on exit - fix exit error code when crash is found (would have exited with 0) changes that affect the normal redis test suite: - refactor check_valgrind_errors into two functions one to search and one to report - move the search half into util.tcl to serve the cluster tests too - ignore "address range perms" valgrind warnings which seem non relevant.
-
Oran Agra authored
in some cases a command that returns an error possibly due to a timing issue causes the tcl code to crash and thus prevents the rest of the tests from running. this adds an option to make the test proceed despite the crash. maybe it should be the default mode some day.
-
Oran Agra authored
reduce code duplication in aof.tcl. move creation of clients into the test so that it can be skipped
-
Oran Agra authored
-
Oran Agra authored
- skip full units - skip a single test (not just a list of tests) - when skipping tag, skip spinning up servers, not just the tests - skip tags when running against an external server too - allow using multiple tags (split them)
-
Oran Agra authored
this is important when running a test with --loop
-
Oran Agra authored
-
- 03 Sep, 2020 1 commit
-
-
Oran Agra authored
During long running scripts or loading RDB/AOF, we may need to do some defragging. Since processEventsWhileBlocked is called periodically at unknown intervals, and many cron jobs either depend on run_with_period (including active defrag), or rely on being called at server.hz rate (i.e. active defrag knows ho much time to run by looking at server.hz), the whileBlockedCron may have to run a loop triggering the cron jobs in it (currently only active defrag) several times. Other changes: - Adding a test for defrag during aof loading. - Changing key-load-delay config to take negative values for fractions of a microsecond sleep
-
- 30 Aug, 2020 1 commit
-
-
Yossi Gottlieb authored
Don't assume `ps` handles `-h` to display output without headers and manually trim headers line from output.
-
- 27 Aug, 2020 2 commits
-
-
valentinogeron authored
If the server gets MULTI command followed by only read commands, and right before it gets the EXEC it reaches OOM, the client will get OOM response. So, from now on, it will get OOM response only if there was at least one command that was tagged with `use-memory` flag
-
Oran Agra authored
plus minor other fixes to list.tcl
-
- 23 Aug, 2020 2 commits
-
-
Valentino Geron authored
When calling to LPOS command when RANK is higher than matches, the return value is non valid response. For example: ``` LPUSH l a :1 LPOS l b RANK 5 COUNT 10 *-4 ``` It may break client-side parser. Now, we count how many replies were replied in the array. ``` LPUSH l a :1 LPOS l b RANK 5 COUNT 10 *0 ```
-
Yossi Gottlieb authored
-
- 20 Aug, 2020 1 commit
-
-
杨博东 authored
After fork, the child process(redis-aof-rewrite) will get the fd opened by the parent process(redis), when redis killed by kill -9, it will not graceful exit(call prepareForShutdown()), so redis-aof-rewrite thread may still alive, the fd(lock) will still be held by redis-aof-rewrite thread, and redis restart will fail to get lock, means fail to start. This issue was causing failures in the cluster tests in github actions. Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 17 Aug, 2020 1 commit
-
-
Yossi Gottlieb authored
The `REDISMODULE_CLIENTINFO_FLAG_SSL` flag was already a part of the `RedisModuleClientInfo` structure but was not implemented.
-
- 12 Aug, 2020 2 commits
-
-
Yossi Gottlieb authored
Add Linux kernel OOM killer control option. This adds the ability to control the Linux OOM killer oom_score_adj parameter for all Redis processes, depending on the process role (i.e. master, replica, background child). A oom-score-adj global boolean flag control this feature. In addition, specific values can be configured using oom-score-adj-values if additional tuning is required.
-
Mota authored
-
- 11 Aug, 2020 1 commit
-
-
Tyson Andre authored
This is a rebased version of #3078 originally by shaharmor with the following patches by TysonAndre made after rebasing to work with the updated C API: 1. Add 2 more unit tests (wrong argument count error message, integer over 64 bits) 2. Use addReplyArrayLen instead of addReplyMultiBulkLen. 3. Undo changes to src/help.h - for the ZMSCORE PR, I heard those should instead be automatically generated from the redis-doc repo if it gets updated Motivations: - Example use case: Client code to efficiently check if each element of a set of 1000 items is a member of a set of 10 million items. (Similar to reasons for working on #7593) - HMGET and ZMSCORE already exist. This may lead to developers deciding to implement functionality that's best suited to a regular set with a data type of sorted set or hash map instead, for the multi-get support. Currently, multi commands or lua scripting to call sismember multiple times would almost definitely be less efficient than a native smismember for the following reasons: - Need to fetch the set from the string every time instead of reusing the C pointer. - Using pipelining or multi-commands would result in more bytes sent and received by the client for the repeated SISMEMBER KEY sections. - Need to specially encode the data and decode it from the client for lua-based solutions. - Proposed solutions using Lua or SADD/SDIFF could trigger writes to memory, which is undesirable on a redis replica server or when commands get replicated to replicas. Co-Authored-By:
Shahar Mor <shahar@peer5.com> Co-Authored-By:
Tyson Andre <tysonandre775@hotmail.com>
-
- 09 Aug, 2020 2 commits
-
-
Meir Shpilraien (Spielrein) authored
Added RedisModule_HoldString that either returns a shallow copy of the given String (by increasing the String ref count) or a new deep copy of String in case its not possible to get a shallow copy. Co-authored-by:
Itamar Haber <itamar@redislabs.com>
-
Oran Agra authored
When runtest-cluster, at first, we need to create a cluster use spawn_instance, a port which is not used is choosen, however sometimes we can't run server on the port. possibley due to a race with another process taking it first. such as redis/redis/runs/896537490. It may be due to the machine problem or In order to reduce the probability of failure when start redis in runtest-cluster, we attemp to use another port when find server do not start up. Co-authored-by:
Oran Agra <oran@redislabs.com> Co-authored-by:
yanhui13 <yanhui13@meituan.com>
-
- 06 Aug, 2020 1 commit
-
-
Oran Agra authored
Diskless master has some inherent latencies. 1) fork starts with delay from cron rather than immediately 2) replica is put online only after an ACK. but the ACK was sent only once a second. 3) but even if it would arrive immediately, it will not register in case cron didn't yet detect that the fork is done. Besides that, when a replica disconnects, it doesn't immediately attempts to re-connect, it waits for replication cron (one per second). in case it was already online, it may be important to try to re-connect as soon as possible, so that the backlog at the master doesn't vanish. In case it disconnected during rdb transfer, one can argue that it's not very important to re-connect immediately, but this is needed for the "diskless loading short read" test to be able to run 100 iterations in 5 seconds, rather than 3 (waiting for replication cron re-connection) changes in this commit: 1) sync command starts a fork immediately if no sync_delay is configured 2) replica sends REPLCONF ACK when done reading the rdb (rather than on 1s cron) 3) when a replica unexpectedly disconnets, it immediately tries to re-connect rather than waiting 1s 4) when when a child exits, if there is another replica waiting, we spawn a new one right away, instead of waiting for 1s replicationCron. 5) added a call to connectWithMaster from replicationSetMaster. which is called from the REPLICAOF command but also in 3 places in cluster.c, in all of these the connection attempt will now be immediate instead of delayed by 1 second. side note: we can add a call to rdbPipeReadHandler in replconfCommand when getting a REPLCONF ACK from the replica to solve a race where the replica got the entire rdb and EOF marker before we detected that the pipe was closed. in the test i did see this race happens in one about of some 300 runs, but i concluded that this race is unlikely in real life (where the replica is on another host and we're more likely to first detect the pipe was closed. the test runs 100 iterations in 3 seconds, so in some cases it'll take 4 seconds instead (waiting for another REPLCONF ACK). Removing unneeded startBgsaveForReplication from updateSlavesWaitingForBgsave Now that CheckChildrenDone is calling the new replicationStartPendingFork (extracted from serverCron) there's actually no need to call startBgsaveForReplication from updateSlavesWaitingForBgsave anymore, since as soon as updateSlavesWaitingForBgsave returns, CheckChildrenDone is calling replicationStartPendingFork that handles that anyway. The code in updateSlavesWaitingForBgsave had a bug in which it ignored repl-diskless-sync-delay, but removing that code shows that this bug was hiding another bug, which is that the max_idle should have used >= and not >, this one second delay has a big impact on my new test.
-
- 04 Aug, 2020 3 commits
-
-
WuYunlong authored
-
Tyson Andre authored
Syntax: `ZMSCORE KEY MEMBER [MEMBER ...]` This is an extension of #2359 amended by Tyson Andre to work with the changed unstable API, add more tests, and consistently return an array. - It seemed as if it would be more likely to get reviewed after updating the implementation. Currently, multi commands or lua scripting to call zscore multiple times would almost definitely be less efficient than a native ZMSCORE for the following reasons: - Need to fetch the set from the string every time instead of reusing the C pointer. - Using pipelining or multi-commands would result in more bytes sent by the client for the repeated `ZMSCORE KEY` sections. - Need to specially encode the data and decode it from the client for lua-based solutions. - The fastest solution I've seen for large sets(thousands or millions) involves lua and a variadic ZADD, then a ZINTERSECT, then a ZRANGE 0 -1, then UNLINK of a temporary set (or lua). This is still inefficient. Co-authored-by:
Tyson Andre <tysonandre775@hotmail.com>
-
Oran Agra authored
apparenlty on github actions sometimes 500ms is not enough
-
- 31 Jul, 2020 1 commit
-
-
Oran Agra authored
besides, hooks test was time sensitive. when the replica managed to reconnect quickly after the client kill, the test would fail
-
- 30 Jul, 2020 1 commit
-
-
WuYunlong authored
-
- 28 Jul, 2020 2 commits
-
-
Yossi Gottlieb authored
Fix consistency test added in af5167b7 without considering TLS redis-cli configuration.
-
Oran Agra authored
- the test now waits for specific set of log messages rather than wait for timeout looking for just one message. - we don't wanna sample the current length of the log after an action, due to a race, we need to start the search from the line number of the last message we where waiting for. - when attempting to trigger a full sync, use multi-exec to avoid a race where the replica manages to re-connect before we completed the set of actions that should force a full sync. - fix verify_log_message which was broken and unused
-