1. 03 Sep, 2020 1 commit
    • Oran Agra's avatar
      Run active defrag while blocked / loading (#7726) · 9ef8d2f6
      Oran Agra authored
      During long running scripts or loading RDB/AOF, we may need to do some
      defragging. Since processEventsWhileBlocked is called periodically at
      unknown intervals, and many cron jobs either depend on run_with_period
      (including active defrag), or rely on being called at server.hz rate
      (i.e. active defrag knows ho much time to run by looking at server.hz),
      the whileBlockedCron may have to run a loop triggering the cron jobs in it
      (currently only active defrag) several times.
      
      Other changes:
      - Adding a test for defrag during aof loading.
      - Changing key-load-delay config to take negative values for fractions
        of a microsecond sleep
      9ef8d2f6
  2. 02 Sep, 2020 3 commits
  3. 01 Sep, 2020 1 commit
  4. 28 Aug, 2020 2 commits
    • maohuazhu's avatar
      Optimize __ziplistCascadeUpdate algorithm (#6886) · ee4a15aa
      maohuazhu authored
      
      
      The previous algorithm is of O(n^2) time complexity.
      It would have run through the ziplist entries one by one, each time doing a `realloc` and a
      `memmove` (moving the entire tail of the ziplist).
      
      The new algorithm is O(n), it runs over all the records once, computing the size of the `realloc`
      needed, then does one `realloc`, and run thought the records again doing many smaller `memmove`s,
      each time moving just one record.
      
      So this change reduces many reallocs, and moves each record just once.
      Co-authored-by: default avatarzhumaohua <zhumaohua@megvii.com>
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      ee4a15aa
    • Jim Brunner's avatar
      Use H/W Monotonic clock and updates to AE (#7644) · c01e94a4
      Jim Brunner authored
      Update adds a general source for retrieving a monotonic time.
      In addition, AE has been updated to utilize the new monotonic
      clock for timer processing.
      
      This performance improvement is **not** enabled in a default build due to various H/W compatibility
      concerns, see README.md for details. It does however change the default use of gettimeofday with
      clock_gettime and somewhat improves performance.
      
      This update provides the following
      1. An interface for retrieving a monotonic clock. getMonotonicUs returns a uint64_t (aka monotime)
         with the number of micro-seconds from an arbitrary point. No more messing with tv_sec/tv_usec.
         Simple routines are provided for measuring elapsed milli-seconds or elapsed micro-seconds (the
         most common use case for a monotonic timer). No worries about time moving backwards.
      2. High-speed assembler implementation for x86 and ARM. The standard method for retrieving the
         monotonic clock is POSIX.1b (1993): cloc...
      c01e94a4
  5. 27 Aug, 2020 3 commits
    • Oran Agra's avatar
      Fix rejectCommand trims newline in shared error objects, hung clients (#7714) · 9fcd9e19
      Oran Agra authored
      65a3307b
      
       (released in 6.0.6) has a side effect, when processCommand
      rejects a command with pre-made shared object error string, it trims the
      newlines from the end of the string. if that string is later used with
      addReply, the newline will be missing, breaking the protocol, and
      leaving the client hung.
      
      It seems that the only scenario which this happens is when replying with
      -LOADING to some command, and later using that reply from the CONFIG
      SET command (still during loading). this will result in hung client.
      
      Refactoring the code in order to avoid trimming these newlines from
      shared string objects, and do the newline trimming only in other cases
      where it's needed.
      Co-authored-by: default avatarGuy Benoish <guy.benoish@redislabs.com>
      9fcd9e19
    • Oran Agra's avatar
      Update memory metrics for INFO during loading (#7690) · 8bdcbbb0
      Oran Agra authored
      During a long AOF or RDB loading, the memory stats were not updated, and
      INFO would return stale data, specifically about fragmentation and RSS.
      In the past some of these were sampled directly inside the INFO command,
      but were moved to cron as an optimization.
      
      This commit introduces a concept of loadingCron which should take
      some of the responsibilities of serverCron.
      It attempts to limit it's rate to approximately the server Hz, but may
      not be very accurate.
      
      In order to avoid too many system call, we use the cached ustime, and
      also make sure to update it in both AOF loading and RDB loading inside
      processEventsWhileBlocked (it seems AOF loading was missing it).
      8bdcbbb0
    • valentinogeron's avatar
      EXEC with only read commands should not be rejected when OOM (#7696) · b7289e91
      valentinogeron authored
      If the server gets MULTI command followed by only read
      commands, and right before it gets the EXEC it reaches OOM,
      the client will get OOM response.
      
      So, from now on, it will get OOM response only if there was
      at least one command that was tagged with `use-memory` flag
      b7289e91
  6. 25 Aug, 2020 2 commits
    • filipe oliveira's avatar
      Extended redis-benchmark instant metrics and overall latency report (#7600) · 21784def
      filipe oliveira authored
      A first step to enable a consistent full percentile analysis on query latency so that we can fully understand the performance and stability characteristics of the redis-server system we are measuring. It also improves the instantaneous reported metrics, and the csv output format.
      21784def
    • Itamar Haber's avatar
      Expands lazyfree's effort estimate to include Streams (#5794) · 5b0a06af
      Itamar Haber authored
      Otherwise, it is treated as a single allocation and freed synchronously. The following logic is used for estimating the effort in constant-ish time complexity:
      
      1. Check the number of nodes.
      1. Add an allocation for each consumer group registered inside the stream.
      1. Check the number of PELs in the first CG, and then add this count times the number of CGs.
      1. Check the number of consumers in the first CG, and then add this count times the number of CGs.
      5b0a06af
  7. 24 Aug, 2020 2 commits
  8. 23 Aug, 2020 2 commits
  9. 21 Aug, 2020 1 commit
  10. 20 Aug, 2020 6 commits
  11. 19 Aug, 2020 1 commit
  12. 18 Aug, 2020 5 commits
  13. 17 Aug, 2020 2 commits
    • Yossi Gottlieb's avatar
      Module API: fix missing RM_CLIENTINFO_FLAG_SSL. (#7666) · 64c360c5
      Yossi Gottlieb authored
      The `REDISMODULE_CLIENTINFO_FLAG_SSL` flag was already a part of the `RedisModuleClientInfo` structure but was not implemented.
      64c360c5
    • Yossi Gottlieb's avatar
      TLS: relax verification on CONFIG SET. (#7665) · fb2a94af
      Yossi Gottlieb authored
      Avoid re-configuring (and validating) SSL/TLS configuration on `CONFIG
      SET` when TLS is not actively enabled for incoming connections, cluster
      bus or replication.
      
      This fixes failures when tests run without `--tls` on binaries that were
      built with TLS support.
      
      An additional benefit is that it's now possible to perform a multi-step
      configuration process while TLS is disabled. The new configuration will
      be verified and applied only when TLS is effectively enabled.
      fb2a94af
  14. 15 Aug, 2020 1 commit
  15. 14 Aug, 2020 1 commit
    • Nathan Scott's avatar
      Annotate module API functions in redismodule.h for use with -fno-common (#6900) · 11cd983d
      Nathan Scott authored
      
      
      In order to keep the redismodule.h self-contained but still usable with
      gcc v10 and later, annotate each API function tentative definition with
      the __common__ attribute.  This avoids the 'multiple definition' errors
      modules will otherwise see for all API functions at link time.
      
      Further details at gcc.gnu.org/gcc-10/porting_to.html
      
      Turn the existing __attribute__ ((unused)), ((__common__)) and ((print))
      annotations into conditional macros for any compilers not accepting this
      syntax.  These macros only expand to API annotations under gcc.
      
      Provide a pre- and post- macro for every API function, so that they can
      be defined differently by the file that includes redismodule.h.
      
      Removing REDISMODULE_API_FUNC in the interest of keeping the function
      declarations readable.
      Co-authored-by: default avatarYossi Gottlieb <yossigo@gmail.com>
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      11cd983d
  16. 13 Aug, 2020 1 commit
    • caozb's avatar
      wait command optimization (#7333) · d10b2f31
      caozb authored
      
      
      Client that issued WAIT last will most likely have the highest replication offset, so imagine a probably common case where all clients are waiting for the same number of replicas. we prefer the loop to start from the last client (one waiting for the highest offset), so that the optimization in the function will call replicationCountAcksByOffset for each client until it found a good one, and stop calling it for the rest of the clients.
      the way the loop was implemented would mean that in such case it is likely to call replicationCountAcksByOffset for all clients.
      
      Note: the change from > to >= is not directly related to the above.
      Co-authored-by: default avatar曹正斌 <caozb@jiedaibao.com>
      d10b2f31
  17. 12 Aug, 2020 5 commits
  18. 11 Aug, 2020 1 commit