- 09 Jun, 2021 1 commit
-
-
Yossi Gottlieb authored
This commit revives the improves the ability to run the test suite against external servers, instead of launching and managing `redis-server` processes as part of the test fixture. This capability existed in the past, using the `--host` and `--port` options. However, it was quite limited and mostly useful when running a specific tests. Attempting to run larger chunks of the test suite experienced many issues: * Many tests depend on being able to start and control `redis-server` themselves, and there's no clear distinction between external server compatible and other tests. * Cluster mode is not supported (resulting with `CROSSSLOT` errors). This PR cleans up many things and makes it possible to run the entire test suite against an external server. It also provides more fine grained controls to handle cases where the external server supports a subset of the Redis commands, limited number of databases, cluster mode, etc. The tests directory now contains a `README.md` file that describes how this works. This commit also includes additional cleanups and fixes: * Tests can now be tagged. * Tag-based selection is now unified across `start_server`, `tags` and `test`. * More information is provided about skipped or ignored tests. * Repeated patterns in tests have been extracted to common procedures, both at a global level and on a per-test file basis. * Cleaned up some cases where test setup was based on a previous test executing (a major anti-pattern that repeats itself in many places). * Cleaned up some cases where test teardown was not part of a test (in the future we should have dedicated teardown code that executes even when tests fail). * Fixed some tests that were flaky running on external servers.
-
- 07 Jun, 2021 2 commits
-
-
Fabian Eichinger authored
Till now GET and NX were mutually exclusive. This change make their combination mean a "Get or Set" command. If the key exists it returns the old value and avoids setting, and if it does't exist it returns nil and sets it to the new value (possibly with expiry time)
-
Huang Zhw authored
The decision to stop trimming due to LIMIT in XADD and XTRIM was after the limit was reached. i.e. the code was deleting **at least** that count of records (from the LIMIT argument's perspective, not the MAXLEN), instead of **up to** that count of records. see #9046
-
- 01 Jun, 2021 2 commits
-
-
Oran Agra authored
-
Oran Agra authored
The test that was merged yesterday fails with valgrind and freebsd CI that are too slow, and 10 seconds apparently passed between the time the command was sent to redis and the time it was actually executed. ``` *** [err]: All TTL in commands are propagated as absolute timestamp in replication stream in tests/unit/expire.tcl Expected 'del a' to match 'set foo1 bar PXAT *' (context: type source line 778 file /home/runner/work/redis/redis/tests/test_helper.tcl cmd {assert_match [lindex $patterns $j] [read_from_replication_stream $s]} proc ::assert_replication_stream level 1) ```
-
- 30 May, 2021 1 commit
-
-
ny0312 authored
Till now, on replica full-sync we used to transfer absolute time for TTL, however when a command arrived (EXPIRE or EXPIREAT), we used to propagate it as is to replicas (possibly with relative time), but always translate it to EXPIREAT (absolute time) to AOF. This commit changes that and will always use absolute time for propagation. see discussion in #8433 Furthermore, we Introduce new commands: `EXPIRETIME/PEXPIRETIME` that allow extracting the absolute TTL time from a key.
-
- 20 May, 2021 1 commit
-
-
YaacovHazan authored
When test stop 'load handler' by killing the process that generating the load, some commands that already in the input buffer, still might be processed by the server. This may cause some instability in tests, that count on that no more commands processed after we stop the `load handler' In this commit, new proc 'wait_load_handlers_disconnected' added, to verify that no more cammands from any 'load handler' prossesed, by checking that the clients who genreate the load is disconnceted. Also, replacing check of dbsize with wait_for_ofs_sync before comparing debug digest, as it would fail in case the last key the workload wrote was an overridden key (not a new one). Affected tests Race fix: - failover command to specific replica works - Connect multiple replicas at the same time (issue #141), master diskless=$mdl, replica diskless=$sdl - AOF rewrite during write load: RDB preamble=$rdbpre Cleanup and speedup: - Test replication with blocking lists and sorted sets operations - Test replication with parallel clients writing in different DBs - Test replication partial resync: $descr (diskless: $mdl, $sdl, reconnect: $reconnect
-
- 19 May, 2021 2 commits
-
-
Madelyn Olson authored
Redact commands that include sensitive data from slowlog and monitor
-
Oran Agra authored
I recently saw this failure: [err]: lazy free a stream with all types of metadata in tests/unit/lazyfree.tcl Expected '2' to be equal to '1' (context: type eval line 23 cmd {assert_equal [s lazyfreed_objects] 1} proc ::test) The only explanation for such a thing is that the async flushdb wasn't done before we did the resetstat
-
- 18 May, 2021 1 commit
-
-
Oran Agra authored
-
- 17 May, 2021 1 commit
-
-
Oran Agra authored
And also add tests to cover lazy free of streams with various types of metadata (see #8932)
-
- 13 May, 2021 1 commit
-
-
Raghav Muddur authored
* EVALSHA_RO and EVAL_RO Commands Added new readonly versions of EVAL and EVALSHA.
-
- 04 May, 2021 1 commit
-
-
yoav-steinberg authored
When client breached the output buffer soft limit but then went idle, we didn't disconnect on soft limit timeout, now we do. Note this also resolves some sporadic test failures in due to Linux buffering data which caused tests to fail if during the test we went back under the soft COB limit. Co-authored-by:
Oran Agra <oran@redislabs.com> Co-authored-by:
sundb <sundbcn@gmail.com>
-
- 27 Apr, 2021 1 commit
-
-
Huang Zhw authored
Use an invalid IP address to trigger CONFIG SET bind failure, instead of DNS which is not guaranteed to always fail.
-
- 20 Apr, 2021 1 commit
-
-
Madelyn Olson authored
Interior rax pointers were not being freed
-
- 19 Apr, 2021 4 commits
-
-
Hanna Fadida authored
Adding a new type mask for key space notification, REDISMODULE_NOTIFY_MODULE, to enable unique notifications from commands on REDISMODULE_KEYTYPE_MODULE type keys (which is currently unsupported). Modules can subscribe to a module key keyspace notification by RM_SubscribeToKeyspaceEvents, and clients by notify-keyspace-events of redis.conf or via the CONFIG SET, with the characters 'd' or 'A' (REDISMODULE_NOTIFY_MODULE type mask is part of the '**A**ll' notation for key space notifications). Refactor: move some pubsub test infra from pubsub.tcl to util.tcl to be re-used by other tests.
-
guybe7 authored
Before this commit using RM_Call without "!" could cause the master to lazy-expire a key (delete it) but without replicating to replicas. This could cause the replica's memory usage to gradually grow and could also cause consistency issues if the master and replica have a clock diff. This bug was introduced in #8617 Added a test which demonstrates that scenario.
-
Harkrishn Patro authored
In the initial release of Redis 6.2 setting a user to only allow pubsub access to a specific channel, and doing ACL SAVE, resulted in an assertion when ACL LOAD was used. This was later changed by #8723 (not yet released), but still not properly resolved (now it errors instead of crash). The problem is that the server that generates an ACL file, doesn't know what would be the setting of the acl-pubsub-default config in the server that will load it. so ACL SAVE needs to always start with resetchannels directive. This should still be compatible with old acl files (from redis 6.0), and ones from earlier versions of 6.2 that didn't mess with channels. Co-authored-by:
Harkrishn Patro <harkrisp@amazon.com> Co-authored-by:
Oran Agra <oran@redislabs.com>
-
sundb authored
The tail size of c->reply is 16kb, but in the test only publish a few chars each time, due to a change in #8699, the obuf limit is now checked a new memory allocation is made, so this test would have sometimes failed to trigger a soft limit disconnection in time. The solution is to write bigger payloads to the output buffer, but still limit their rate (not more than 100k/s).
-
- 14 Apr, 2021 1 commit
-
-
YaacovHazan authored
replace the hardcoded after 2000, with waiting for the sync and wait for condition
-
- 13 Apr, 2021 1 commit
-
-
Oran Agra authored
-
- 07 Apr, 2021 1 commit
-
-
Yang Bodong authored
Fix out of range error messages to be clearer (avoid mentioning 9223372036854775807) * Fix XAUTOCLAIM COUNT option confusing error msg * Fix other RPOP and alike error message to mention positive
-
- 05 Apr, 2021 1 commit
-
-
Huang Zhw authored
Fix "default" and overwritten / reset users will not have pubsub channels permissions by default. (#8723) Background: Redis 6.2 added ACL control for pubsub channels (#7993), which were supposed to be permissive by default to retain compatibility with redis 6.0 ACL. But due to a bug, only newly created users got this `acl-pubsub-default` applied, while overwritten (updated) users got reset to `resetchannels` (denied). Since the "default" user exists before loading the config file, any ACL change to it, results in an update / overwrite. So when a "default" user is loaded from config file or include ACL file with no channels related rules, the user will not have any permissions to any channels. But other users will have default permissions to any channels. When upgraded from 6.0 with config rewrite, this will lead to "default" user channels permissions lost. When users are loaded from include file, then call "acl load", users will also lost channels permissions. Similarly, the `reset` ACL rule, would have reset the user to be denied access to any channels, ignoring `acl-pubsub-default` and breaking compatibility with redis 6.0. The implication of this fix is that it regains compatibility with redis 6.0, but breaks compatibility with redis 6.2.0 and 2.0.1. e.g. after the upgrade, the default user will regain access to pubsub channels. Other changes: Additionally this commit rename server.acl_pubusub_default to server.acl_pubsub_default and fix typo in acl tests.
-
- 01 Apr, 2021 3 commits
-
-
Valentino Geron authored
This command used to return the last scanned entry id as the cursor, instead of the next one to be scanned. so in the next call, the user could / should have sent `(cursor` and not just `cursor` if he wanted to avoid scanning the same record twice. Scanning the record twice would look odd if someone is checking what exactly was scanned, but it also has a side effect of incrementing the delivery count twice.
-
guybe7 authored
If GT/LT fails the operation we need to reply with nill (like failure due to NX). Other changes: Add the missing $encoding suffix to many zset tests Note: there's a behavior change just in case of INCR + GT/LT that fails. The old code was replying with the wrong (rejected) score, and now it'll reply with nil. Note that that's anyway a corner case so this "behavior change" shouldn't have too much affect. Using GT/LT with INCR has a predictable result even before we run the command (INCR GT will only only / always fail if the increment is negative).
-
sundb authored
Problem: Currently, when performing random distribution verification, we determine the probability of each element occurring in the sum, but the probability is only an estimate, these tests had rare sporadic failures, and we cannot verify what the probability of failure will be. Solution: Using the chi-square distribution instead of the original random distribution validation makes the test more reasonable and easier to find problems.
-
- 30 Mar, 2021 1 commit
-
-
JunhuaY authored
the bug was also discussed in #8716, and was solved in #8719, but incompletely: when the server is started, and the save option is default, if you issue the " config set save "" " to change the save option, and then issue the “config rewrite” command, the " save "" " won't be saved.
-
- 29 Mar, 2021 2 commits
-
-
Yossi Gottlieb authored
-
Meir Shpilraien (Spielrein) authored
'processCommandAndResetClient' returns 1 if client is dead. It does it by checking if serve.current_client is NULL. On script timeout, Redis will re-enter 'processCommandAndResetClient' and when finish we will set server.current_client to NULL. This will cause later to falsely return 1 and think that the client that sent the timed-out script is dead (Redis to stop reading from the client buffer).
-
- 26 Mar, 2021 1 commit
-
-
Huang Zhw authored
Add publish channel permissions check in processCommand. processCommand didn't check publish channel permissions, so we can queue a publish command in a transaction. But when exec the transaction, it will fail with -NOPERM. We also union keys/commands/channels permissions check togegher in ACLCheckAllPerm. Remove pubsubCheckACLPermissionsOrReply in publishCommand/subscribeCommand/psubscribeCommand. Always check permissions in processCommand/execCommand/ luaRedisGenericCommand.
-
- 25 Mar, 2021 1 commit
-
-
Oran Agra authored
* SLOWLOG didn't record anything for blocked commands because the client was reset and argv was already empty. there was a fix for this issue specifically for modules, now it works for all blocked clients. * The original command argv (before being re-written) was also reset before adding the slowlog on behalf of the blocked command. * Latency monitor is now updated regardless of the slowlog flags of the command or its execution (their purpose is to hide sensitive info from the slowlog, not hide the fact the latency happened). * Latency monitor now uses real_cmd rather than c->cmd (which may be different if the command got re-written, e.g. GEOADD) Changes: * Unify shared code between slowlog insertion in call() and updateStatsOnUnblock(), hopefully prevent future bugs from happening due to the later being overlooked. * Reset CLIENT_PREVENT_LOGGING in resetClient rather than after command processing. * Add a test for SLOWLOG and BLPOP Notes: - real_cmd == c->lastcmd, except inside MULTI and Lua. - blocked commands never happen in these cases (MULTI / Lua) - real_cmd == c->cmd, except for when the command is rewritten (e.g. GEOADD) - blocked commands (currently) are never rewritten - other than the command's CLIENT_PREVENT_LOGGING, and the execution flag CLIENT_PREVENT_LOGGING, other cases that we want to avoid slowlog are on AOF loading (specifically CMD_CALL_SLOWLOG will be off when executed from execCommand that runs from an AOF)
-
- 24 Mar, 2021 1 commit
-
-
yoav-steinberg authored
This fixes a race where a bgsave can start during the test after we verified no bgsave is running.
-
- 22 Mar, 2021 1 commit
-
-
Yossi Gottlieb authored
-
- 17 Mar, 2021 1 commit
-
-
Meir Shpilraien (Spielrein) authored
pcall function runs another LUA function in protected mode, this means that any error will be caught by this function and will not stop the LUA execution. The script kill mechanism uses error to stop the running script. Scripts that uses pcall can catch the error raise by the script kill mechanism, this will cause a script like this to be unkillable: local f = function() while 1 do redis.call('ping') end end while 1 do pcall(f) end The fix is, when we want to kill the script, we set the hook function to be invoked after each line. This will promise that the execution will get another error before it is able to enter the pcall function again.
-
- 16 Mar, 2021 1 commit
-
-
Madelyn Olson authored
Redact config set requirepass/masterauth/masteruser from slowlog in addition to showing ACL commands without sensitive values.
-
- 15 Mar, 2021 1 commit
-
-
guybe7 authored
1. moduleReplicateMultiIfNeeded should use server.in_eval like moduleHandlePropagationAfterCommandCallback 2. server.in_eval could have been set to 1 and not reset back to 0 (a lot of missed early-exits after in_eval is already 1) Note: The new assertions in processCommand cover (2) and I added two module tests to cover (1) Implications: If an EVAL that failed (and thus left server.in_eval=1) runs before a module command that replicates, the replication stream will contain MULTI (because moduleReplicateMultiIfNeeded used to check server.lua_caller which is NULL at this point) but not EXEC (because server.in_eval==1) This only affects modules as module.c the only user of server.in_eval. Affects versions 6.2.0, 6.2.1
-
- 11 Mar, 2021 1 commit
-
-
Harkrishn Patro authored
Co-authored-by:
Harkrishn Patro <harkrisp@amazon.com>
-
- 10 Mar, 2021 1 commit
-
-
guybe7 authored
Bug 1: When a module ctx is freed moduleHandlePropagationAfterCommandCallback is called and handles propagation. We want to prevent it from propagating commands that were not replicated by the same context. Example: 1. module1.foo does: RM_Replicate(cmd1); RM_Call(cmd2); RM_Replicate(cmd3) 2. RM_Replicate(cmd1) propagates MULTI and adds cmd1 to also_propagagte 3. RM_Call(cmd2) create a new ctx, calls call() and destroys the ctx. 4. moduleHandlePropagationAfterCommandCallback is called, calling alsoPropagates EXEC (Note: EXEC is still not written to socket), setting server.in_trnsaction = 0 5. RM_Replicate(cmd3) is called, propagagting yet another MULTI (now we have nested MULTI calls, which is no good) and then cmd3 We must prevent RM_Call(cmd2) from resetting server.in_transaction. REDISMODULE_CTX_MULTI_EMITTED was revived for that purpose. Bug 2: Fix issues with nested RM_Call where some have '!' and some don't. Example: 1. module1.foo does RM_Call of module2.bar without replication (i.e. no '!') 2. module2.bar internally calls RM_Call of INCR with '!' 3. at the end of module1.foo we call RM_ReplicateVerbatim We want the replica/AOF to see only module1.foo and not the INCR from module2.bar Introduced a global replication_allowed flag inside RM_Call to determine whether we need to replicate or not (even if '!' was specified) Other changes: Split beforePropagateMultiOrExec to beforePropagateMulti afterPropagateExec just for better readability
-
- 08 Mar, 2021 2 commits
-
-
Yossi Gottlieb authored
-
Yossi Gottlieb authored
It seems like non-Linux sockets may be less greedy, resulting with more transient client output buffers. Haven't proven this but empirically when stressing this test on non-Linux tends to exhibit increased mem_clients_normal values.
-