1. 12 Jan, 2021 1 commit
  2. 27 Oct, 2020 12 commits
    • Oran Agra's avatar
      INFO report peak memory before eviction (#7894) · 851ec530
      Oran Agra authored
      In some cases one command added a very big bulk of memory, and this
      would be "resolved" by the eviction before the next command.
      
      Seeing an unexplained mass eviction we would wish to
      know the highest momentary usage too.
      
      Tracking it in call() and beforeSleep() adds some hooks in AOF and RDB
      loading.
      
      The fix in clientsCronTrackExpansiveClients is related to #7874
      
      (cherry picked from commit 457b7073)
      851ec530
    • guybe7's avatar
      Minor improvements to module blocked on keys (#7903) · e4e24460
      guybe7 authored
      - Clarify some documentation comments
      - Make sure blocked-on-keys client privdata is accessible
        from withing the timeout callback
      - Handle blocked clients in beforeSleep - In case a key
        becomes "ready" outside of processCommand
      
      See #7879 #7880
      
      (cherry picked from commit addf47dc)
      e4e24460
    • Yossi Gottlieb's avatar
      Introduce getKeysResult for getKeysFromCommand. · 945983ac
      Yossi Gottlieb authored
      Avoid using a static buffer for short key index responses, and make it
      caller's responsibility to stack-allocate a result type. Responses that
      don't fit are still allocated on the heap.
      
      (cherry picked from commit 9b7f8ba8)
      945983ac
    • Oran Agra's avatar
      memory reporting of clients argv (#7874) · 76f3a63d
      Oran Agra authored
      track and report memory used by clients argv.
      this is very usaful in case clients started sending a command and didn't
      complete it. in which case the first args of the command are already
      trimmed from the query buffer.
      
      in an effort to avoid cache misses and overheads while keeping track of
      these, i avoid calling sdsZmallocSize and instead use the sdslen /
      bulk-len which can at least give some insight into the problem.
      
      This memory is now added to the total clients memory usage, as well as
      the client list.
      
      (cherry picked from commit bea40e6a)
      76f3a63d
    • Oran Agra's avatar
      Include internal sds fragmentation in MEMORY reporting (#7864) · 12055ed8
      Oran Agra authored
      The MEMORY command is used for debugging memory usage, so it should include internal
      fragmentation, same as used_memory
      
      (cherry picked from commit eb6241a3)
      12055ed8
    • Wang Yuan's avatar
      Don't write replies if close the client ASAP (#7202) · 757ad7cd
      Wang Yuan authored
      
      
      Before this commit, we would have continued to add replies to the reply buffer even if client
      output buffer limit is reached, so the used memory would keep increasing over the configured limit.
      What's more, we shouldn’t write any reply to the client if it is set 'CLIENT_CLOSE_ASAP' flag
      because that doesn't conform to its definition and we will close all clients flagged with
      'CLIENT_CLOSE_ASAP' in ‘beforeSleep’.
      
      Because of code execution order, before this, we may firstly write to part of the replies to
      the socket before disconnecting it, but in fact, we may can’t send the full replies to clients
      since OS socket buffer is limited. But this unexpected behavior makes some commands work well,
      for instance ACL DELUSER, if the client deletes the current user, we need to send reply to client
      and close the connection, but before, we close the client firstly and write the reply to reply
      buffer. secondly, we shouldn't do this despite the fact it works well in most cases.
      
      We add a flag 'CLIENT_CLOSE_AFTER_COMMAND' to mark clients, this flag means we will close the
      client after executing commands and send all entire replies, so that we can write replies to
      reply buffer during executing commands, send replies to clients, and close them later.
      
      We also fix some implicit problems. If client output buffer limit is enforced in 'multi/exec',
      all commands will be executed completely in redis and clients will not read any reply instead of
      partial replies. Even more, if the client executes 'ACL deluser' the using user in 'multi/exec',
      it will not read the replies after 'ACL deluser' just like before executing 'client kill' itself
      in 'multi/exec'.
      
      We added some tests for output buffer limit breach during multi-exec and using a pipeline of
      many small commands rather than one with big response.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      (cherry picked from commit 57709c4b)
      757ad7cd
    • WuYunlong's avatar
      Make main thread killable so that it can be canceled at any time. · 4832cf4f
      WuYunlong authored
      Refine comment of makeThreadKillable().
      
      This commit can be backported to 5.0, only if we also backport 8b70cb0e
      
      .
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      (cherry picked from commit 647cac5b)
      4832cf4f
    • Oran Agra's avatar
      RM_GetContextFlags provides indication that we're in a fork child (#7783) · fd610ae0
      Oran Agra authored
      
      (cherry picked from commit 2458e548)
      fd610ae0
    • Wang Yuan's avatar
      Remove tmp rdb file in background thread (#7762) · 0bdddd3c
      Wang Yuan authored
      We're already using bg_unlink in several places to delete the rdb file in the background,
      and avoid paying the cost of the deletion from our main thread.
      This commit uses bg_unlink to remove the temporary rdb file in the background too.
      
      However, in case we delete that rdb file just before exiting, we don't actually wait for the
      background thread or the main thread to delete it, and just let the OS clean up after us.
      i.e. we open the file, unlink it and exit with the fd still open.
      
      Furthermore, rdbRemoveTempFile can be called from a thread and was using snprintf which is
      not async-signal-safe, we now use ll2string instead.
      
      (cherry picked from commit b002d2b4)
      0bdddd3c
    • WuYunlong's avatar
      bio: fix doFastMemoryTest. · 5d933226
      WuYunlong authored
      If one thread got SIGSEGV, function sigsegvHandler() would be triggered,
      it would call bioKillThreads(). But call pthread_cancel() to cancel itself
      would make it block. Also note that if SIGSEGV is caught by bio thread, it
      should kill the main thread in order to give a positive report.
      
      (cherry picked from commit 8b70cb0e)
      5d933226
    • Wang Yuan's avatar
      Remove dead global variable 'lru_clock' (#7782) · 3efdbbc7
      Wang Yuan authored
      
      (cherry picked from commit 86511bbb)
      3efdbbc7
    • Oran Agra's avatar
      Squash merging 125 typo/grammar/comment/doc PRs (#7773) · 2ae7f491
      Oran Agra authored
      List of squashed commits or PRs
      ===============================
      
      commit 66801ea
      Author: hwware <wen.hui.ware@gmail.com>
      Date:   Mon Jan 13 00:54:31 2020 -0500
      
          typo fix in acl.c
      
      commit 46f55db
      Author: Itamar Haber <itamar@redislabs.com>
      Date:   Sun Sep 6 18:24:11 2020 +0300
      
          Updates a couple of comments
      
          Specifically:
      
          * RM_AutoMemory completed instead of pointing to docs
          * Updated link to custom type doc
      
      commit 61a2aa0
      Author: xindoo <xindoo@qq.com>
      Date:   Tue Sep 1 19:24:59 2020 +0800
      
          Correct errors in code comments
      
      commit a5871d1
      Author: yz1509 <pro-756@qq.com>
      Date:   Tue Sep 1 18:36:06 2020 +0800
      
          fix typos in module.c
      
      commit 41eede7
      Author: bookug <bookug@qq.com>
      Date:   Sat Aug 15 01:11:33 2020 +0800
      
          docs: fix typos in comments
      
      commit c303c84
      Author: lazy-snail <ws.niu@outlook.com>
      Date:   Fri Aug 7 11:15:44 2020 +0800
      
          fix spelling in redis.conf
      
      commit 1eb76bf
      Author: zhujian <zhujianxyz@gmail.com>...
      2ae7f491
  3. 10 Sep, 2020 3 commits
  4. 01 Sep, 2020 10 commits
  5. 20 Jul, 2020 3 commits
    • dmurnane's avatar
      Notify systemd on sentinel startup (#7168) · 29b20fd5
      dmurnane authored
      
      Co-authored-by: default avatarDaniel Murnane <dmurnane@eitccorp.com>
      (cherry picked from commit 9242ccf2)
      29b20fd5
    • Oran Agra's avatar
      EXEC always fails with EXECABORT and multi-state is cleared · 05e483cb
      Oran Agra authored
      In order to support the use of multi-exec in pipeline, it is important that
      MULTI and EXEC are never rejected and it is easy for the client to know if the
      connection is still in multi state.
      
      It was easy to make sure MULTI and DISCARD never fail (done by previous
      commits) since these only change the client state and don't do any actual
      change in the server, but EXEC is a different story.
      
      Since in the past, it was possible for clients to handle some EXEC errors and
      retry the EXEC, we now can't affort to return any error on EXEC other than
      EXECABORT, which now carries with it the real reason for the abort too.
      
      Other fixes in this commit:
      - Some checks that where performed at the time of queuing need to be re-
        validated when EXEC runs, for instance if the transaction contains writes
        commands, it needs to be aborted. there was one check that was already done
        in execCommand (-READONLY), but other checks where missing: -OOM, -MISCONF,
        -NOREPLICAS, -MASTERDOWN
      - When a command is rejected by processCommand it was rejected with addReply,
        which was not recognized as an error in case the bad command came from the
        master. this will enable to count or MONITOR these errors in the future.
      - make it easier for tests to create additional (non deferred) clients.
      - add tests for the fixes of this commit.
      
      (cherry picked from commit 65a3307b)
      05e483cb
    • Tomasz Poradowski's avatar
      ensure SHUTDOWN_NOSAVE in Sentinel mode · 3328b7a5
      Tomasz Poradowski authored
      - enforcing of SHUTDOWN_NOSAVE flag in one place to make it consitent
        when running in Sentinel mode
      
      (cherry picked from commit 4ee011ad)
      3328b7a5
  6. 12 Jun, 2020 2 commits
  7. 09 Jun, 2020 1 commit
    • Oran Agra's avatar
      Avoid rejecting WATCH / UNWATCH, like MULTI/EXEC/DISCARD · b35fdf1d
      Oran Agra authored
      Much like MULTI/EXEC/DISCARD, the WATCH and UNWATCH are not actually
      operating on the database or server state, but instead operate on the
      client state. the client may send them all in one long pipeline and check
      all the responses only at the end, so failing them may lead to a
      mismatch between the client state on the server and the one on the
      client end, and execute the wrong commands (ones that were meant to be
      discarded)
      
      the watched keys are not actually stored in the client struct, but they
      are in fact part of the client state. for instance, they're not cleared
      or moved in SWAPDB or FLUSHDB.
      b35fdf1d
  8. 28 May, 2020 1 commit
    • antirez's avatar
      Remove the meaningful offset feature. · 2112a570
      antirez authored
      After a closer look, the Redis core devleopers all believe that this was
      too fragile, caused many bugs that we didn't expect and that were very
      hard to track. Better to find an alternative solution that is simpler.
      2112a570
  9. 14 May, 2020 4 commits
    • antirez's avatar
      Track events processed while blocked globally. · 74249be4
      antirez authored
      Related to #7234.
      74249be4
    • antirez's avatar
      Some rework of #7234. · 8bf660af
      antirez authored
      8bf660af
    • Oran Agra's avatar
      fix redis 6.0 not freeing closed connections during loading. · 9da134cd
      Oran Agra authored
      This bug was introduced by a recent change in which readQueryFromClient
      is using freeClientAsync, and despite the fact that now
      freeClientsInAsyncFreeQueue is in beforeSleep, that's not enough since
      it's not called during loading in processEventsWhileBlocked.
      furthermore, afterSleep was called in that case but beforeSleep wasn't.
      
      This bug also caused slowness sine the level-triggered mode of epoll
      kept signaling these connections as readable causing us to keep doing
      connRead again and again for ll of these, which keep accumulating.
      
      now both before and after sleep are called, but not all of their actions
      are performed during loading, some are only reserved for the main loop.
      
      fixes issue #7215
      9da134cd
    • antirez's avatar
      Tracking: send eviction messages when evicting entries. · e3b5648d
      antirez authored
      A fix for #7249.
      e3b5648d
  10. 08 May, 2020 3 commits