1. 05 Nov, 2020 3 commits
    • Yossi Gottlieb's avatar
      Fix crash log output on ARM. (#8020) · 7e4325cb
      Yossi Gottlieb authored
      7e4325cb
    • Oran Agra's avatar
      Better INFO fields to track diskless and disk-based replication progress (#7981) · 7ace7231
      Oran Agra authored
      Expose new `loading_rdb_used_mem` showing the used memory of the server
      that saved the RDB file we're currently using.
      This is useful in diskless replication when the total size of the rdb is
      unkown, and can be used as a rought estimation of progres.
      
      Use that new field to calculate the "user friendly"
      `loading_loaded_perc` and `loading_eta_seconds`.
      
      Expose `master_sync_total_bytes` and `master_sync_total_bytes` to complement
      on the existing `master_sync_total_bytes` (which cannot be used on its own
      to calculate progress).
      
      Add "user friendly" field for `master_sync_perc`
      7ace7231
    • Yossi Gottlieb's avatar
      Add RESET command. (#7982) · 1fd456f9
      Yossi Gottlieb authored
      
      
      Perform full reset of all client connection states, is if the client was
      disconnected and re-connected. This affects:
      
      * MULTI state
      * Watched keys
      * MONITOR mode
      * Pub/Sub subscription
      * ACL/Authenticated state
      * Client tracking state
      * Cluster read-only/asking state
      * RESP version (reset to 2)
      * Selected database
      * CLIENT REPLY state
      
      The response is +RESET to make it easily distinguishable from other
      responses.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      Co-authored-by: default avatarItamar Haber <itamar@redislabs.com>
      1fd456f9
  2. 04 Nov, 2020 5 commits
    • Tomasz Poradowski's avatar
      rdb: corrected RedisModuleIO initialization point (#8014) · d8fbd3a8
      Tomasz Poradowski authored
      
      
      - rdbSaveSingleModuleAux() used RedisModuleIO's "bytes" field for
        tracking written bytes before calling moduleInitIOContext() which sets
        "bytes" to zero
      - rdbSaveObject() re-initialized RedisModuleIO too late
      
      This return value is not used at the moment since it's only tested
      against -1, and the actual byte count isn't used yet.
      Co-authored-by: default avatarTomasz Poradowski <tomasz.poradowski@generiscorp.com>
      d8fbd3a8
    • filipe oliveira's avatar
      Enable specifying TLS ciphers(suites) in redis-cli/redis-benchmark (#8005) · 10b50069
      filipe oliveira authored
      
      
      Enable specifying the preferred ciphers and/or ciphersuites for redis-cli/redis-benchmark.
      Co-authored-by: default avatarYossi Gottlieb <yossigo@gmail.com>
      10b50069
    • Egor Seredin's avatar
      Allow '\0' inside of result of sdscatvprintf, and efficiency improvements (#6260) · f4ca3d87
      Egor Seredin authored
      This will allow to use: RedisModule_CreateStringPrintf(ctx, "%s %c %s", "string1", 0, "string2");
      
      On large string, the previous code would incrementally retry to double the output buffer.
      now it uses the the return value of snprintf and grows to the right size in one step.
      
      and also avoids an excessive strlen in sdscat at the end.
      f4ca3d87
    • Wen Hui's avatar
      redis-cli cluster import support source and target that need auth (#7994) · 639b73cd
      Wen Hui authored
      Make it possible for redis-cli cluster import to work with source and
      target that require AUTH.
      
      Adding two different flags --cluster-from-user, --cluster-from-pass
      and --cluster-askpass for source node authentication.
      Also for target authentication, using existing --user and --pass flag.
      
      Example:
      
      ./redis-cli --cluster import 127.0.0.1:7000 --cluster-from 127.0.0.1:6379 --pass 1234 --user default --cluster-from-user default --cluster-from-pass 123456
      
      ./redis-cli --cluster import 127.0.0.1:7000 --cluster-from 127.0.0.1:6379 --askpass --cluster-from-user default --cluster-from-askpass
      639b73cd
    • Oran Agra's avatar
      Add maxclients and cluster_connections to INFO CLIENTS (#7979) · a698a639
      Oran Agra authored
      Few config settings are also reflected by the INFO command.
      these are mainly ones that are important for either an instant view of
      the server status (to compare a metric to it's limit config),
      Important configurations that are necessary in the crash log (which
      currently doesn't print the config),
      And things that are important for monitoring solutions (such as
      Prometheus), which rely on INFO to collect their data.
      
      Add cluster_connections to INFO CLUSTER:
      This makes it possible to be combined together with connected_clients
      and connected_slaves and be matched against maxclients
      a698a639
  3. 03 Nov, 2020 3 commits
  4. 02 Nov, 2020 1 commit
    • guybe7's avatar
      Modules: Improve timer accuracy (#7987) · 1a91a270
      guybe7 authored
      The bug occurs when 'callback' re-registers itself to a point
      in the future and the execution time in non-negligible:
      'now' refers to time BEFORE callback was executed and is used
      to calculate 'next_period'.
      We must get the actual current time when calculating 'next_period'
      1a91a270
  5. 28 Oct, 2020 7 commits
    • yoav-steinberg's avatar
      Add local address to CLIENT LIST, and a CLIENT KILL filter. (#7913) · 84b3c18f
      yoav-steinberg authored
      Useful when you want to know through which bind address the client connected to
      the server in case of multiple bind addresses.
      
      - Adding `laddr` field to CLIENT list showing the local (bind) address.
      - Adding `LADDR` option to CLIENT KILL to kill all the clients connected
        to a specific local address.
      - Refactoring to share code.
      84b3c18f
    • Oran Agra's avatar
      Optionally (default) fail to start if requested bind address is not available (#7936) · 441bfa2d
      Oran Agra authored
      Background:
      #3467 (redis 4.0.0), started ignoring ENOPROTOOPT, but did that only for
      the default bind (in case bind config wasn't explicitly set).
      #5598 (redis 5.0.3), added that for bind addresses explicitly set
      (following bug reports in Debian for redis 4.0.9 and 5.0.1), it
      also ignored a bunch of other errors like EPROTONOSUPPORT which was
      requested in #3894, and also added EADDRNOTAVAIL (wasn't clear why).
      
      This (ignoring EADDRNOTAVAIL) makes redis start successfully, even if a
      certain network interface isn't up yet , in which case we rather redis
      fail and will be re-tried when the NIC is up, see #7933.
      
      However, it turns out that when IPv6 is disabled (supported but unused),
      the error we're getting is EADDRNOTAVAIL. and in many systems the
      default config file tries to bind to localhost for both v4 and v6 and
      would like to silently ignore the error on v6 if disabled.
      This means that we sometimes want to ignore EADDRNOTAVAIL and other times
      we wanna fail.
      
      So this commit changes these main things:
      1. Ignore all the errors we ignore for both explicitly requested bind
         address and a default implicit one.
      2. Add a '-' prefix to allow EADDRNOTAVAIL be ignored (by default that's
         different than the previous behavior).
      3. Restructure that function in a more readable and maintainable way see
         below.
      4. Make the default behavior of listening to all achievable by setting
        a bind config directive to * (previously only possible by omitting
        it)
      5. document everything.
      
      The old structure of this function was that even if there are no bind
      addresses requested, the loop that runs though the bind addresses runs
      at least once anyway!
      In that one iteration of the loop it binds to both v4 and v6 addresses,
      handles errors for each of them separately, and then eventually at the
      if-else chain, handles the error of the last bind attempt again!
      This was very hard to read and very error prone to maintain, instead now
      when the bind info is missing we create one with two entries, and run
      the simple loop twice.
      441bfa2d
    • Madelyn Olson's avatar
      d310beb4
    • Madelyn Olson's avatar
      411bcf1a
    • Wen Hui's avatar
    • sundb's avatar
      69871760
    • filipe oliveira's avatar
      TLS Support for redis-benchmark (#7959) · 39436b21
      filipe oliveira authored
      39436b21
  6. 27 Oct, 2020 6 commits
    • WuYunlong's avatar
      Fix waste of CPU time about server log in serverCron. · 66037309
      WuYunlong authored
      When all the work is just adding logs, we could pull
      the condition out so as to use less CPU time when
      loglevel is bigger than LL_VERBOSE.
      66037309
    • Oran Agra's avatar
      Fix cluster access to unaligned memory (SIGBUS on old ARM) (#7958) · 380f6048
      Oran Agra authored
      Turns out this was broken since version 4.0 when we added sds size
      classes.
      The cluster code uses sds for the receive buffer, and then casts it to a
      struct and accesses a 64 bit variable.
      This commit replaces the use of sds with a simple reallocated buffer.
      380f6048
    • zhenwei pi's avatar
      Disable THP if enabled (#7381) · a9c06021
      zhenwei pi authored
      In case redis starts and find that THP is enabled ("always"), instead
      of printing a log message, which might go unnoticed, redis will try to
      disable it (just for the redis process).
      
      Note: it looks like on self-bulit kernels THP is likely be set to "always" by default.
      
      Some discuss about THP side effect on Linux:
      according to http://www.antirez.com/news/84
      
      , we can see that
      redis latency spikes are caused by linux kernel THP feature.
      I have tested on E3-2650 v3, and found that 2M huge page costs
      about 0.25ms to fix COW page fault.
      
      Add a new config 'disable-thp', the recommended setting is 'yes',
      (default) the redis tries to disable THP by prctl syscall. But
      users who really want THP can set it to "no"
      
      Thanks to Oran & Yossi for suggestions.
      Signed-off-by: default avatarzhenwei pi <pizhenwei@bytedance.com>
      a9c06021
    • Andrij Fedyk's avatar
      rdb.c: fix typo in a comment (#7970) · 825fe7bd
      Andrij Fedyk authored
      825fe7bd
    • WuYunlong's avatar
      Speedup cluster failover. (#7948) · 7fa56dd7
      WuYunlong authored
      
      
      This commit deals with manual failover as well as non-manual failover.
      
      We did tests with manual failover as follows:
      1, Setup redis cluster which holds 16 partions, each having only
         1 corresponding replica.
      2, Write a batch of data to redis cluster and make sure the redis is doing
         a active expire in serverCron.
      3, Do a manual failover sequentially to each partions with a time interval
         of 3 minutes.
      4, Collect logs and do some computaiton work.
      
      The result:
      case    avgTime    maxTime    minTime
      C1      95.8ms	   227ms      25ms
      C2      47.9ms     96ms       12ms
      C3      12.6ms     27ms       7ms
      
      Explanation
      case C1: All nodes use the version before optimization
      case C2: Masters use the elder version while replicas use the optimized version
      case C3: All nodes use the optimized version
      failover time: The time between when replica got a `manual failover request` and
                     when it `won the failover election`.
      avgTime: average failover time
      maxTime: maximum failover time
      minTime: mimimum failover time
      ms: millisecond
      Co-authored-by: default avatarchendq8 <c.d_q@163.com>
      7fa56dd7
    • Madelyn Olson's avatar
      Only supress implitic fallthrough on GCC 7 · dac26729
      Madelyn Olson authored
      dac26729
  7. 26 Oct, 2020 2 commits
    • Yossi Gottlieb's avatar
      Fix wrong zmalloc_size() assumption. (#7963) · 9824fe3e
      Yossi Gottlieb authored
      When using a system with no malloc_usable_size(), zmalloc_size() assumed
      that the heap allocator always returns blocks that are long-padded.
      
      This may not always be the case, and will result with zmalloc_size()
      returning a size that is bigger than allocated. At least in one case
      this leads to out of bound write, process crash and a potential security
      vulnerability.
      
      Effectively this does not affect the vast majority of users, who use
      jemalloc or glibc.
      
      This problem along with a (different) fix was reported by Drew DeVault.
      9824fe3e
    • filipe oliveira's avatar
      redis-benchmark: add tests, --version, a minor bug fixes (#7947) · 01acfa71
      filipe oliveira authored
      
      
      - add test suite coverage for redis-benchmark
      - add --version (similar to what redis-cli has)
      - fix bug sending more requests than intended when pipeline > 1.
      - when done sending requests, avoid freeing client in the write handler, in theory before
        responses are received (probably dead code since the read handler will call clientDone first)
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      01acfa71
  8. 25 Oct, 2020 2 commits
  9. 23 Oct, 2020 1 commit
  10. 22 Oct, 2020 2 commits
    • Wen Hui's avatar
      do not add save parameter during config rewrite in sentinel mode (#7945) · 0f370f9b
      Wen Hui authored
      Previous code would have added default redis save parameters
      to the config file on rewrite, which would have been silently ignored
      when the config file is loaded.
      
      The new code avoids adding this, and also actively removes these lines
      If added by a previous config rewrite. 
      0f370f9b
    • Qu Chen's avatar
      WATCH no longer ignores keys which have expired for MULTI/EXEC. (#7920) · 556acefe
      Qu Chen authored
      This wrong behavior was backed by a test, and also documentation, and dates back to 2010.
      But it makes no sense to anyone involved so it was decided to change that.
      
      Note that 20eeddfb (invalidate watch on expire on access) was released in 6.0 RC2
      and 2d1968f8 released in in 6.0.0 GA (invalidate watch when key is evicted).
      both of which do similar changes.
      556acefe
  11. 20 Oct, 2020 2 commits
  12. 19 Oct, 2020 2 commits
    • Wen Hui's avatar
      fix double fclose in aofrewrite (#7919) · 04a0af90
      Wen Hui authored
      minor fix for a bug which happen on error handling code
      and doesn't look like it could have caused any real harm
      (fd number wouldn't have been reused yet)
      04a0af90
    • Wen Hui's avatar
      Support ACL for Sentinel Mode (#7888) · 0047702a
      Wen Hui authored
      
      
      This commit implements ACL for Sentinel mode, main work of this PR includes:
      
      - Update Sentinel command table in order to better support ACLs.
      - Fix couple of things which currently blocks the support for ACL on sentinel mode.
      - Provide "sentinel sentinel-user" and "sentinel sentinel-pass " configuration in order to let sentinel authenticate with a specific user in other sentinels.
      - requirepass is kept just for compatibility with old config files
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      0047702a
  13. 18 Oct, 2020 4 commits
    • Oran Agra's avatar
      INFO report peak memory before eviction (#7894) · 457b7073
      Oran Agra authored
      In some cases one command added a very big bulk of memory, and this
      would be "resolved" by the eviction before the next command.
      
      Seeing an unexplained mass eviction we would wish to
      know the highest momentary usage too.
      
      Tracking it in call() and beforeSleep() adds some hooks in AOF and RDB
      loading.
      
      The fix in clientsCronTrackExpansiveClients is related to #7874
      457b7073
    • Wen Hui's avatar
      support NOMKSTREAM option in xadd command (#7910) · f328194d
      Wen Hui authored
      introduces a NOMKSTREAM option for xadd command, this would be useful for some
      use cases when we do not want to create new stream by default:
      
      XADD key [MAXLEN [~|=] <count>] [NOMKSTREAM] <ID or *> [field value] [field value]
      f328194d
    • Pierre Jambet's avatar
      t_set.c comment update for srandmemberWithCountCommand (#7922) · f6010e10
      Pierre Jambet authored
      Reference the correct "case", case 4, in the comment explaining the need
      for case 3, when the number of request items is too close to the
      cardinality of the set. Case 4 is indeed the "natural approach"
      referenced earlier in that sentence.
      f6010e10
    • Tommy Joe Lund's avatar
      Fix typo in server.h (#7921) · 786d6d55
      Tommy Joe Lund authored
      786d6d55