1. 27 Apr, 2022 3 commits
    • filipe oliveira's avatar
      Optimization: Avoid deferred array reply on ZRANGE commands BYRANK (#10337) · ef78757e
      filipe oliveira authored
      
      
      Avoid deferred array reply on genericZrangebyrankCommand() when consumer type is client.
      I.e. any ZRANGE / ZREVRNGE (when tank is used).
      This was a performance regression introduced in #7844 (v 6.2) mainly affecting pipelined workloads.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      (cherry picked from commit 1dc89e2d)
      ef78757e
    • filipe oliveira's avatar
      Optimize deferred replies to use shared objects instead of sprintf (#10334) · 35e697de
      filipe oliveira authored
      
      
      Avoid sprintf/ll2string on setDeferredAggregateLen()/addReplyLongLongWithPrefix() when we can used shared objects.
      In some pipelined workloads this achieves about 10% improvement.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      (cherry picked from commit b857928b)
      35e697de
    • Oran Agra's avatar
      Fix and improve module error reply statistics (#10278) · d6a8e64e
      Oran Agra authored
      This PR handles several aspects
      1. Calls to RM_ReplyWithError from thread safe contexts don't violate thread safety.
      2. Errors returning from RM_Call to the module aren't counted in the statistics (they
        might be handled silently by the module)
      3. When a module propagates a reply it got from RM_Call to it's client, then the error
        statistics are counted.
      
      This is done by:
      1. When appending an error reply to the output buffer, we avoid updating the global
        error statistics, instead we cache that error in a deferred list in the client struct.
      2. When creating a RedisModuleCallReply object, the deferred error list is moved from
        the client into that object.
      3. when a module calls RM_ReplyWithCallReply we copy the deferred replies to the dest
        client (if that's a real client, then that's when the error statistics are updated to the server)
      
      Note about RM_ReplyWithCallReply: if the original reply had an array with errors, and the module
      replied with just a portion of the original reply, and not the entire reply, the errors are currently not
      propagated and the errors stats will not get propagated.
      
      Fix #10180
      
      (cherry picked from commit b099889a)
      d6a8e64e
  2. 04 Oct, 2021 3 commits
    • yvette903's avatar
      Fix: client pause uses an old timeout (#9477) · 23ed37dd
      yvette903 authored
      A write request may be paused unexpectedly because `server.client_pause_end_time` is old.
      
      **Recreate this:**
      redis-cli -p 6379
      127.0.0.1:6379> client pause 500000000 write
      OK
      127.0.0.1:6379> client unpause
      OK
      127.0.0.1:6379> client pause 10000 write
      OK
      127.0.0.1:6379> set key value
      
      The write request `set key value` is paused util  the timeout of 500000000 milliseconds was reached.
      
      **Fix:**
      reset `server.client_pause_end_time` = 0 in `unpauseClients`
      
      (cherry picked from commit f560531d)
      23ed37dd
    • zhaozhao.zz's avatar
      Fix wrong offset when replica pause (#9448) · 9b25484a
      zhaozhao.zz authored
      When a replica paused, it would not apply any commands event the command comes from master, if we feed the non-applied command to replication stream, the replication offset would be wrong, and data would be lost after failover(since replica's `master_repl_offset` grows but command is not applied).
      
      To fix it, here are the changes:
      * Don't update replica's replication offset or propagate commands to sub-replicas when it's paused in `commandProcessed`.
      * Show `slave_read_repl_offset` in info reply.
      * Add an assert to make sure master client should never be blocked unless pause or module (some modules may use block way to do background (parallel) processing and forward original block module command to the replica, it's not a good way but it can work, so the assert excludes module now, but someday in future all modules should rewrite block command to propagate like what `BLPOP` does).
      
      (cherry picked from commit 1b83353d)
      9b25484a
    • Oran Agra's avatar
      Prevent unauthenticated client from easily consuming lots of memory (CVE-2021-32675) · 757f8f77
      Oran Agra authored
      This change sets a low limit for multibulk and bulk length in the
      protocol for unauthenticated connections, so that they can't easily
      cause redis to allocate massive amounts of memory by sending just a few
      characters on the network.
      The new limits are 10 arguments of 16kb each (instead of 1m of 512mb)
      757f8f77
  3. 21 Jul, 2021 3 commits
    • Yossi Gottlieb's avatar
      Fix CLIENT UNBLOCK crashing modules. (#9167) · 79fa5618
      Yossi Gottlieb authored
      Modules that use background threads with thread safe contexts are likely
      to use RM_BlockClient() without a timeout function, because they do not
      set up a timeout.
      
      Before this commit, `CLIENT UNBLOCK` would result with a crash as the
      `NULL` timeout callback is called. Beyond just crashing, this is also
      logically wrong as it may throw the module into an unexpected client
      state.
      
      This commits makes `CLIENT UNBLOCK` on such clients behave the same as
      any other client that is not in a blocked state and therefore cannot be
      unblocked.
      
      (cherry picked from commit aa139e2f)
      79fa5618
    • Oran Agra's avatar
      Test infra, handle RESP3 attributes and big-numbers and bools (#9235) · 6cd84b64
      Oran Agra authored
      - promote the code in DEBUG PROTOCOL to addReplyBigNum
      - DEBUG PROTOCOL ATTRIB skips the attribute when client is RESP2
      - networking.c addReply for push and attributes generate assertion when
        called on a RESP2 client, anything else would produce a broken
        protocol that clients can't handle.
      
      (cherry picked from commit 6a5bac30)
      6cd84b64
    • yvette903's avatar
      Fix coredump after Client Unpause command when threaded I/O is enabled (#9041) · 8bd5e391
      yvette903 authored
      Fix crash when using io-threads-do-reads and issuing CLIENT PAUSE and
      CLIENT UNPAUSE.
      This issue was introduced in redis 6.2 together with the FAILOVER command.
      
      (cherry picked from commit 096c5fd5)
      8bd5e391
  4. 01 Jun, 2021 2 commits
  5. 15 Apr, 2021 2 commits
    • guybe7's avatar
      Add a timeout mechanism for replicas stuck in fullsync (#8762) · d63d0260
      guybe7 authored
      Starting redis 6.0 (part of the TLS feature), diskless master uses pipe from the fork
      child so that the parent is the one sending data to the replicas.
      This mechanism has an issue in which a hung replica will cause the master to wait
      for it to read the data sent to it forever, thus preventing the fork child from terminating
      and preventing the creations of any other forks.
      
      This PR adds a timeout mechanism, much like the ACK-based timeout,
      we disconnect replicas that aren't reading the RDB file fast enough.
      d63d0260
    • Bonsai's avatar
  6. 13 Apr, 2021 1 commit
  7. 06 Apr, 2021 1 commit
  8. 31 Mar, 2021 1 commit
  9. 30 Mar, 2021 1 commit
  10. 29 Mar, 2021 1 commit
  11. 25 Mar, 2021 1 commit
    • Oran Agra's avatar
      Fix SLOWLOG for blocked commands (#8632) · 497351ad
      Oran Agra authored
      * SLOWLOG didn't record anything for blocked commands because the client
        was reset and argv was already empty. there was a fix for this issue
        specifically for modules, now it works for all blocked clients.
      * The original command argv (before being re-written) was also reset
        before adding the slowlog on behalf of the blocked command.
      * Latency monitor is now updated regardless of the slowlog flags of the
        command or its execution (their purpose is to hide sensitive info from
        the slowlog, not hide the fact the latency happened).
      * Latency monitor now uses real_cmd rather than c->cmd (which may be
        different if the command got re-written, e.g. GEOADD)
      
      Changes:
      * Unify shared code between slowlog insertion in call() and
        updateStatsOnUnblock(), hopefully prevent future bugs from happening
        due to the later being overlooked.
      * Reset CLIENT_PREVENT_LOGGING in resetClient rather than after command
        processing.
      * Add a test for SLOWLOG and BLPOP
      
      Notes:
      - real_cmd == c->lastcmd, except inside MULTI and Lua.
      - blocked commands never happen in these cases (MULTI / Lua)
      - real_cmd == c->cmd, except for when the command is rewritten (e.g.
        GEOADD)
      - blocked commands (currently) are never rewritten
      - other than the command's CLIENT_PREVENT_LOGGING, and the
        execution flag CLIENT_PREVENT_LOGGING, other cases that we want to
        avoid slowlog are on AOF loading (specifically CMD_CALL_SLOWLOG will
        be off when executed from execCommand that runs from an AOF)
      497351ad
  12. 17 Mar, 2021 1 commit
    • Theo Buehler's avatar
      Fixes for systems with 64-bit time (#8662) · 169be042
      Theo Buehler authored
      Some operating systems (e.g., NetBSD and OpenBSD) have switched to
      using a 64-bit integer for time_t on all platforms. This results in currently
      harmless compiler warnings due to potential truncation.
      These changes fix these minor portability concerns.
      
      * Fix format string for systems with 64 bit time
      * use llabs to avoid truncation with 64 bit time
      169be042
  13. 16 Mar, 2021 1 commit
  14. 21 Feb, 2021 1 commit
  15. 08 Feb, 2021 1 commit
  16. 02 Feb, 2021 1 commit
  17. 28 Jan, 2021 2 commits
  18. 19 Jan, 2021 2 commits
  19. 13 Jan, 2021 1 commit
  20. 08 Jan, 2021 2 commits
  21. 07 Jan, 2021 1 commit
    • YaacovHazan's avatar
      Refactory fork child related infra, Unify child pid · f9dacf8a
      YaacovHazan authored
      This is a refactory commit, isn't suppose to have any actual impact.
      it does the following:
      - keep just one server struct fork child pid variable instead of 3
      - have one server struct variable indicating the purpose of the current fork
        child.
      - redisFork is now responsible of updating the server struct with the pid,
        which means it can be the one that calls updateDictResizePolicy
      - move child info pipe handling into redisFork instead of having them
        repeated outside
      - there are two classes of fork purposes, mutually exclusive group (AOF, RDB,
        Module), and one that can create several forks to coexist in parallel (LDB,
        but maybe Modules some day too, Module API allows for that).
      - minor fix to killRDBChild:
        unlike killAppendOnlyChild and TerminateModuleForkChild, the killRDBChild
        doesn't clear the pid variable or call wait4, so checkChildrenDone does
        the cleanup for it.
        This commit removes the explicit calls to rdbRemoveTempFile, closeChildInfoPipe,
        updateDictResizePolicy, which didn't do any harm, but where unnecessary.
      f9dacf8a
  22. 06 Jan, 2021 2 commits
    • Wen Hui's avatar
      fix memory leak in processInlineBuffer error handling code (#8295) · cfcd0fa6
      Wen Hui authored
      This code path is normally executed only when v6.0 and above replicates from v2.4
      cfcd0fa6
    • guybe7's avatar
      Add XAUTOCLAIM (#7973) · 714e103a
      guybe7 authored
      
      
      New command: XAUTOCLAIM <key> <group> <consumer> <min-idle-time> <start> [COUNT <count>] [JUSTID]
      
      The purpose is to claim entries from a stale consumer without the usual
      XPENDING+XCLAIM combo which takes two round trips.
      
      The syntax for XAUTOCLAIM is similar to scan: A cursor is returned (streamID)
      by each call and should be used as start for the next call. 0-0 means the scan is complete.
      
      This PR extends the deferred reply mechanism for any bulk string (not just counts)
      
      This PR carries some unrelated test code changes:
      - Renames the term "client" into "consumer" in the stream-cgroups test
      - And also changes DEBUG SLEEP into "after"
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      714e103a
  23. 04 Jan, 2021 1 commit
    • Itamar Haber's avatar
      HELP subcommand, continued (#5531) · 9dcdc7e7
      Itamar Haber authored
      
      
      * man-like consistent long formatting
      * Uppercases commands, subcommands and options
      * Adds 'HELP' to HELP for all
      * Lexicographical order
      * Uses value notation and other .md likeness
      * Moves const char *help to top
      * Keeps it under 80 chars
      * Misc help typos, consistent conjuctioning (i.e return and not returns)
      * Uses addReplySubcommandSyntaxError(c) all over
      Signed-off-by: default avatarItamar Haber <itamar@redislabs.com>
      9dcdc7e7
  24. 31 Dec, 2020 1 commit
    • filipe oliveira's avatar
      Add errorstats info section, Add failed_calls and rejected_calls to commandstats (#8217) · 90b9f08e
      filipe oliveira authored
      This Commit pushes forward the observability on overall error statistics and command statistics within redis-server:
      
      It extends INFO COMMANDSTATS to have
      - failed_calls in - so we can keep track of errors that happen from the command itself, broken by command.
      - rejected_calls - so we can keep track of errors that were triggered outside the commmand processing per se
      
      Adds a new section to INFO, named ERRORSTATS that enables keeping track of the different errors that
      occur within redis ( within processCommand and call ) based on the reply Error Prefix ( The first word
      after the "-", up to the first space ).
      
      This commit also fixes RM_ReplyWithError so that it can be correctly identified as an error reply.
      90b9f08e
  25. 27 Dec, 2020 2 commits
    • Oran Agra's avatar
      Fix memory leaks in error replies due to recent change (#8249) · 049cf8cd
      Oran Agra authored
      Recently efaf09ee started using addReplyErrorSds in place of
      addReplySds the later takes ownership of the string but the former did
      not.
      This introduced memory leaks when a script returns an error to redis,
      and also in clusterRedirectClient (two new usages of
      addReplyErrorSds which was mostly unused till now.
      
      This commit chagnes two thanks.
      1. change addReplyErrorSds to take ownership of the error string.
      2. scripting.c doesn't actually need to use addReplyErrorSds, it's a
      perfect match for addReplyErrorFormat (replaces newlines with spaces)
      049cf8cd
    • zhaozhao.zz's avatar
      Tracking: add CLIENT TRACKINGINFO subcommand (#7309) · 299f9ebf
      zhaozhao.zz authored
      
      
      Add CLIENT TRACKINGINFO subcommand
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      299f9ebf
  26. 25 Dec, 2020 1 commit
  27. 24 Dec, 2020 1 commit