1. 01 Apr, 2021 3 commits
    • Oran Agra's avatar
      Solve sentinel test issue in TLS due to recent tests change. (#8728) · 370ab4c4
      Oran Agra authored
      5629dbe7 added a change that configures the tcp (plaintext) port
      alongside the tls port, this causes the INFO command for tcp_port
      to return that instead of the tls port when running in tls, and that broke
      the sentinel tests that query it.
      
      the fix is to add a method that gets the right port from CONFIG instead
      of relying on the tcp_port info field.
      370ab4c4
    • guybe7's avatar
      zsetAdd: Fix wrong reply in case of INCR and GT/LT (#8717) · 843f769b
      guybe7 authored
      If GT/LT fails the operation we need to reply with
      nill (like failure due to NX).
      
      Other changes:
      Add the missing $encoding suffix to many zset tests
      
      Note: there's a behavior change just in case of INCR + GT/LT that fails.
      The old code was replying with the wrong (rejected) score, and now it'll reply with nil.
      
      Note that that's anyway a corner case so this "behavior change" shouldn't have too much affect.
      Using GT/LT with INCR has a predictable result even before we run the command
      (INCR GT will only only / always fail if the increment is negative).
      843f769b
    • sundb's avatar
      Use chi-square for random distributivity verification in test (#8709) · 569a3f45
      sundb authored
      Problem:
      Currently, when performing random distribution verification, we determine
      the probability of each element occurring in the sum, but the probability is
      only an estimate, these tests had rare sporadic failures, and we cannot verify
      what the probability of failure will be.
      
      Solution:
      Using the chi-square distribution instead of the original random distribution
      validation makes the test more reasonable and easier to find problems.
      569a3f45
  2. 30 Mar, 2021 5 commits
    • Jérôme Loyet's avatar
      Add replica-announced config option (#8653) · 91f4f416
      Jérôme Loyet authored
      The 'sentinel replicas <master>' command will ignore replicas with
      `replica-announced` set to no.
      
      The goal of disabling the config setting replica-announced is to allow ghost
      replicas. The replica is in the cluster, synchronize with its master, can be
      promoted to master and is not exposed to sentinel clients. This way, it is
      acting as a live backup or living ghost.
      
      In addition, to prevent the replica to be promoted as master, set
      replica-priority to 0.
      91f4f416
    • Yossi Gottlieb's avatar
      Cluster migration test cleanup. (#8726) · 6a052af8
      Yossi Gottlieb authored
      * Dump more output on error (always, cluster tests currently have no
      verbose flag).
      * Slow down redis-cli check iteration.
      6a052af8
    • Viktor Söderqvist's avatar
      Add support for plaintext clients in TLS cluster (#8587) · 5629dbe7
      Viktor Söderqvist authored
      The cluster bus is established over TLS or non-TLS depending on the configuration tls-cluster. The client ports distributed in the cluster and sent to clients are assumed to be TLS or non-TLS also depending on tls-cluster.
      
      The cluster bus is now extended to also contain the non-TLS port of clients in a TLS cluster, when available. The non-TLS port of a cluster node, when available, is sent to clients connected without TLS in responses to CLUSTER SLOTS, CLUSTER NODES, CLUSTER SLAVES and MOVED and ASK redirects, instead of the TLS port.
      
      The user was able to override the client port by defining cluster-announce-port. Now cluster-announce-tls-port is added, so the user can define an alternative announce port for both TLS and non-TLS clients.
      
      Fixes #8134
      5629dbe7
    • JunhuaY's avatar
      re-fix config rewrite for empty save directive (#8722) · 28375ff6
      JunhuaY authored
      the bug was also discussed in #8716, and was solved in #8719, but incompletely:
      when the server is started, and the save option is default, if you issue the " config set save "" "
      to change the save option, and then issue the “config rewrite” command, the " save "" " won't be saved.
      28375ff6
    • Oran Agra's avatar
      solve race conditions in psync2-pingoff test (#8720) · cd81dcf1
      Oran Agra authored
      Another test race condition in the macos tests.
      the test was waiting for PINGs to be generated and put on the replication stream,
      but waiting for 1 or 2 seconds doesn't really guarantee that.
      then the test that expected 6 full syncs, found only 4
      cd81dcf1
  3. 29 Mar, 2021 3 commits
  4. 26 Mar, 2021 1 commit
    • Huang Zhw's avatar
      make processCommand check publish channel permissions. (#8534) · e138698e
      Huang Zhw authored
      Add publish channel permissions check in processCommand.
      
      processCommand didn't check publish channel permissions, so we can
      queue a publish command in a transaction. But when exec the transaction,
      it will fail with -NOPERM.
      
      We also union keys/commands/channels permissions check togegher in
      ACLCheckAllPerm. Remove pubsubCheckACLPermissionsOrReply in 
      publishCommand/subscribeCommand/psubscribeCommand. Always 
      check permissions in processCommand/execCommand/
      luaRedisGenericCommand.
      e138698e
  5. 25 Mar, 2021 1 commit
    • Oran Agra's avatar
      Fix SLOWLOG for blocked commands (#8632) · 497351ad
      Oran Agra authored
      * SLOWLOG didn't record anything for blocked commands because the client
        was reset and argv was already empty. there was a fix for this issue
        specifically for modules, now it works for all blocked clients.
      * The original command argv (before being re-written) was also reset
        before adding the slowlog on behalf of the blocked command.
      * Latency monitor is now updated regardless of the slowlog flags of the
        command or its execution (their purpose is to hide sensitive info from
        the slowlog, not hide the fact the latency happened).
      * Latency monitor now uses real_cmd rather than c->cmd (which may be
        different if the command got re-written, e.g. GEOADD)
      
      Changes:
      * Unify shared code between slowlog insertion in call() and
        updateStatsOnUnblock(), hopefully prevent future bugs from happening
        due to the later being overlooked.
      * Reset CLIENT_PREVENT_LOGGING in resetClient rather than after command
        processing.
      * Add a test for SLOWLOG and BLPOP
      
      Notes:
      - real_cmd == c->lastcmd, except inside MULTI and Lua.
      - blocked commands never happen in these cases (MULTI / Lua)
      - real_cmd == c->cmd, except for when the command is rewritten (e.g.
        GEOADD)
      - blocked commands (currently) are never rewritten
      - other than the command's CLIENT_PREVENT_LOGGING, and the
        execution flag CLIENT_PREVENT_LOGGING, other cases that we want to
        avoid slowlog are on AOF loading (specifically CMD_CALL_SLOWLOG will
        be off when executed from execCommand that runs from an AOF)
      497351ad
  6. 24 Mar, 2021 3 commits
  7. 22 Mar, 2021 3 commits
  8. 17 Mar, 2021 1 commit
    • Meir Shpilraien (Spielrein)'s avatar
      Fix script kill to work also on scripts that use pcall (#8661) · 9ae4f5c7
      Meir Shpilraien (Spielrein) authored
      pcall function runs another LUA function in protected mode, this means
      that any error will be caught by this function and will not stop the LUA
      execution. The script kill mechanism uses error to stop the running script.
      Scripts that uses pcall can catch the error raise by the script kill mechanism,
      this will cause a script like this to be unkillable:
      
      local f = function()
              while 1 do
                      redis.call('ping')
              end
      end
      while 1 do
              pcall(f)
      end
      
      The fix is, when we want to kill the script, we set the hook function to be invoked 
      after each line. This will promise that the execution will get another
      error before it is able to enter the pcall function again.
      9ae4f5c7
  9. 16 Mar, 2021 2 commits
    • Huang Zhw's avatar
      When tests exit normally, some processes may still be alive (#8647) · a19c4058
      Huang Zhw authored
      In certain scenario start_server may think it failed to start a redis server
      although it started successfully. in these cases, it'll not terminate it, and
      it'll remain running when the test is over.
      
      In start_server if config doesn't have bind (the minimal.conf in introspection.tcl),
      it will try to bind ipv4 and ipv6. One may success while other fails. It will
      output "Could not create server TCP listening socket".
      wait_server_started uses this message to check whether instance started
      successfully. So it will consider that it failed even though redis started successfully.
      
      Additionally, in some cases it wasn't clear to users why the server exited,
      since the warning message printed to the log, could in some cases be harmless,
      and in some cases fatal.
      
      This PR adds makes a clear distinction between a warning log message and
      a fatal one, and changes the test suite to look for the fatal message.
      a19c4058
    • Madelyn Olson's avatar
      Redact slowlog entries for config with sensitive data. (#8584) · e1d98bca
      Madelyn Olson authored
      Redact config set requirepass/masterauth/masteruser from slowlog in addition to showing ACL commands without sensitive values. 
      e1d98bca
  10. 15 Mar, 2021 1 commit
    • guybe7's avatar
      Missing EXEC on modules propagation after failed EVAL execution (#8654) · dba33a94
      guybe7 authored
      1. moduleReplicateMultiIfNeeded should use server.in_eval like
         moduleHandlePropagationAfterCommandCallback
      2. server.in_eval could have been set to 1 and not reset back
         to 0 (a lot of missed early-exits after in_eval is already 1)
      
      Note: The new assertions in processCommand cover (2) and I added
      two module tests to cover (1)
      
      Implications:
      If an EVAL that failed (and thus left server.in_eval=1) runs before a module
      command that replicates, the replication stream will contain MULTI (because
      moduleReplicateMultiIfNeeded used to check server.lua_caller which is NULL
      at this point) but not EXEC (because server.in_eval==1)
      This only affects modules as module.c the only user of server.in_eval.
      
      Affects versions 6.2.0, 6.2.1
      dba33a94
  11. 12 Mar, 2021 1 commit
  12. 11 Mar, 2021 2 commits
  13. 10 Mar, 2021 1 commit
    • guybe7's avatar
      Fix some issues with modules and MULTI/EXEC (#8617) · 3d0b427c
      guybe7 authored
      Bug 1:
      When a module ctx is freed moduleHandlePropagationAfterCommandCallback
      is called and handles propagation. We want to prevent it from propagating
      commands that were not replicated by the same context. Example:
      1. module1.foo does: RM_Replicate(cmd1); RM_Call(cmd2); RM_Replicate(cmd3)
      2. RM_Replicate(cmd1) propagates MULTI and adds cmd1 to also_propagagte
      3. RM_Call(cmd2) create a new ctx, calls call() and destroys the ctx.
      4. moduleHandlePropagationAfterCommandCallback is called, calling
         alsoPropagates EXEC (Note: EXEC is still not written to socket),
         setting server.in_trnsaction = 0
      5. RM_Replicate(cmd3) is called, propagagting yet another MULTI (now
         we have nested MULTI calls, which is no good) and then cmd3
      
      We must prevent RM_Call(cmd2) from resetting server.in_transaction.
      REDISMODULE_CTX_MULTI_EMITTED was revived for that purpose.
      
      Bug 2:
      Fix issues with nested RM_Call where some have '!' and some don't.
      Example:
      1. module1.foo does RM_Call of module2.bar without replication (i.e. no '!')
      2. module2.bar internally calls RM_Call of INCR with '!'
      3. at the end of module1.foo we call RM_ReplicateVerbatim
      
      We want the replica/AOF to see only module1.foo and not the INCR from module2.bar
      
      Introduced a global replication_allowed flag inside RM_Call to determine
      whether we need to replicate or not (even if '!' was specified)
      
      Other changes:
      Split beforePropagateMultiOrExec to beforePropagateMulti afterPropagateExec
      just for better readability
      3d0b427c
  14. 08 Mar, 2021 2 commits
  15. 04 Mar, 2021 1 commit
    • Yossi Gottlieb's avatar
      Improve redis-cli non-binary safe string handling. (#8566) · 3c7d6a18
      Yossi Gottlieb authored
      * The `redis-cli --scan` output should honor output mode (set explicitly or implicitly), and quote key names when not in raw mode.
        * Technically this is a breaking change, but it should be very minor since raw mode is by default on for non-tty output.
        * It should only affect  TTY output (human users) or non-tty output if `--no-raw` is specified.
      
      * Added `--quoted-input` option to treat all arguments as potentially quoted strings.
      * Added `--quoted-pattern` option to accept a potentially quoted pattern.
      
      Unquoting is applied to potentially quoted input only if single or double quotes are used. 
      
      Fixes #8561, #8563
      3c7d6a18
  16. 02 Mar, 2021 1 commit
  17. 01 Mar, 2021 4 commits
  18. 28 Feb, 2021 1 commit
    • Viktor Söderqvist's avatar
      Shared reusable client for RM_Call() (#8516) · 6122f1c4
      Viktor Söderqvist authored
      A single client pointer is added in the server struct. This is
      initialized by the first RM_Call() and reused for every subsequent
      RM_Call() except if it's already in use, which means that it's not
      used for (recursive) module calls to modules. For these, a new
      "fake" client is created each time.
      
      Other changes:
      * Avoid allocating a dict iterator in pubsubUnsubscribeAllChannels
        when not needed
      6122f1c4
  19. 24 Feb, 2021 2 commits
  20. 23 Feb, 2021 2 commits
    • guybe7's avatar
      Fix race in CONFIG REWRITE sanity (#8536) · f745c018
      guybe7 authored
      server may still be LOADING the RDB when receiving the ping
      f745c018
    • Yossi Gottlieb's avatar
      Fix failed tests on Linux Alpine and add a CI job. (#8532) · 95ea7454
      Yossi Gottlieb authored
      * Remove linux/version.h dependency.
      
      This introduces unnecessary dependencies, and generally not a good idea
      as the platform we build on may be different than the platform we run
      on.
      
      To determine if sync_file_range exists we can simply rely on header file
      hints.
      
      * Fix setproctitle() on libmusl.
      
      The previous ifdef checks were a bit too strict for no apparent
      reason.
      
      * Fix tests failure on Linux with no backtrace.
      
      * Add alpine daily CI job.
      95ea7454