1. 07 Jul, 2022 1 commit
  2. 06 Jul, 2022 2 commits
  3. 05 Jul, 2022 1 commit
  4. 04 Jul, 2022 5 commits
    • Harkrishn Patro's avatar
      9bcdd153
    • Wen Hui's avatar
      Add tests for error messages during slot migrations (#10449) · f620e6ac
      Wen Hui authored
      
      
      * Add tests for error messages during slot migrations
      Co-authored-by: default avatarUbuntu <lucas.guang.yang1@huawei.com>
      Co-authored-by: default avatarMadelyn Olson <madelyneolson@gmail.com>
      f620e6ac
    • Harkrishn Patro's avatar
      Optimize number of realloc syscall during multi/exec flow (#10921) · a3704d4e
      Harkrishn Patro authored
      ## Issue
      During the MULTI/EXEC flow, each command gets queued until the `EXEC`
      command is received and during this phase on every command queue, a
      `realloc` is being invoked. This could be expensive based on the realloc
      behavior (if copy to a new memory location). 
      
      
      ## Solution
      In order to reduce the no. of syscall, couple of optimization I've used.
      
      1. By default, reserve memory for atleast two commands. `MULTI/EXEC` for a
      single command doesn't have any significance. Hence, I believe customer wouldn't use it.
      2. For further reservation, increase the memory allocation in exponent growth (power of 2).
      This reduces the no. of `realloc` call from `N` to `log(N)` times.
      
      ## Other changes:
      
      * Include multi exec queued command array in client memory consumption calculation
      (affects client eviction too)
      a3704d4e
    • Qu Chen's avatar
      Unlock cluster config file upon server shutdown. (#10912) · 33b7ff38
      Qu Chen authored
      Currently in cluster mode, Redis process locks the cluster config file when
      starting up and holds the lock for the entire lifetime of the process.
      When the server shuts down, it doesn't explicitly release the lock on the
      cluster config file. We noticed a problem with restart testing that if you shut down
      a very large redis-server process (i.e. with several hundred GB of data stored),
      it takes the OS a while to free the resources and unlock the cluster config file.
      So if we immediately try to restart the redis server process, it might fail to acquire
      the lock on the cluster config file and fail to come up.
      
      This fix explicitly releases the lock on the cluster config file upon a shutdown rather
      than relying on the OS to release the lock, which is a cleaner and safer approach to
      free up resources acquired. 
      33b7ff38
    • Harkrishn Patro's avatar
      Account sharded pubsub channels memory consumption (#10925) · 0ab885a6
      Harkrishn Patro authored
      Account sharded pubsub channels memory consumption in client memory usage
      computation to accurately evict client based on the set threshold for `maxmemory-clients`.
      0ab885a6
  5. 03 Jul, 2022 4 commits
    • Binbin's avatar
      Set aof rewrite status in some backgroundRewriteDoneHandler errors (#10923) · 679344a2
      Binbin authored
      We should also set aof_lastbgrewrite_status to C_ERR on these
      errors. Because aof rewrite did fail, and we did not finish the
      manifest update. Also maintain the stat_aofrw_consecutive_failures.
      679344a2
    • Yossi Gottlieb's avatar
      Fix TLS issues with large replies (#10909) · 0b645d63
      Yossi Gottlieb authored
      This problem was introduced by 496375fc and seems to more easily reproduce on macOS since OpenSSL writes more frequently return with EAGAIN.
      0b645d63
    • Yossi Gottlieb's avatar
      Fix TLS tests on newer tcl-tls/OpenSSL. (#10910) · 69d55768
      Yossi Gottlieb authored
      Before this commit, TLS tests on Ubuntu 22.04 would fail as dropped
      connections result with an ECONNABORTED error thrown instead of an empty
      read.
      69d55768
    • Binbin's avatar
      Always set server.aof_last_write_errno in aof write error (#10917) · 2ab67677
      Binbin authored
      The `can_log` variable prevents us from outputting too
      many error logs. But it should not include the modification
      of server.aof_last_write_errno.
      
      We are doing this because:
      1. In the short write case, we always set aof_last_write_errno
      to ENOSPC, we don't care the `can_log` flag.
      
      2. And we always set aof_last_write_status to C_ERR in aof write
      error (except for FSYNC_ALWAYS, we exit). So there may be a chance
      that `aof_last_write_errno` is not right.
      
      An innocent bug or just a code cleanup.
      2ab67677
  6. 30 Jun, 2022 1 commit
    • Binbin's avatar
      Add SENTINEL command flag to CLIENT/COMMANDS subcommands (#10904) · 35e836c2
      Binbin authored
      This was harmless because we marked the parent command
      with SENTINEL flag. So the populateCommandTable was ok.
      And we also don't show the flag (SENTINEL and ONLY-SENTNEL)
      in COMMAND INFO.
      
      In this PR, we also add the same CMD_SENTINEL and CMD_ONLY_SENTINEL
      flags check when populating the sub-commands.
      so that in the future it'll be possible to add some sub-commands to sentinel or sentinel-only but not others.
      35e836c2
  7. 29 Jun, 2022 1 commit
  8. 28 Jun, 2022 2 commits
  9. 27 Jun, 2022 1 commit
    • Viktor Söderqvist's avatar
      Add missing REDISMODULE_CLIENTINFO_INITIALIZER (#10885) · 6af02100
      Viktor Söderqvist authored
      The module API docs mentions this macro, but it was not defined (so no one could have used it).
      
      Instead of adding it as is, we decided to add a _V1 macro, so that if / when we some day extend this struct,
      modules that use this API and don't need the extra fields, will still use the old version
      and still be compatible with older redis version (despite being compiled with newer redismodule.h)
      6af02100
  10. 26 Jun, 2022 5 commits
    • RinChanNOW!'s avatar
      Support conversion between `RedisModuleString` and `unsigned long long` (#10889) · 28546373
      RinChanNOW! authored
      
      
      Since the ranges of `unsigned long long` and `long long` are different, we cannot read an
      `unsigned long long` integer from a `RedisModuleString` by `RedisModule_StringToLongLong` . 
      
      So I added two new Redis Module APIs to support the conversion between these two types:
      * `RedisModule_StringToULongLong`
      * `RedisModule_CreateStringFromULongLong`
      Signed-off-by: default avatarRinChanNOWWW <hzy427@gmail.com>
      28546373
    • Binbin's avatar
      redis-server command line arguments allow passing config name and value in the same arg (#10866) · d443e312
      Binbin authored
      This commit has two topics.
      
      ## Passing config name and value in the same arg
      In #10660 (Redis 7.0.1), when we supported the config values that can start with `--` prefix (one of the two topics of that PR),
      we broke another pattern: `redis-server redis.config "name value"`, passing both config name
      and it's value in the same arg, see #10865
      
      This wasn't a intended change (i.e we didn't realize this pattern used to work).
      Although this is a wrong usage, we still like to fix it.
      
      Now we support something like:
      ```
      src/redis-server redis.conf "--maxmemory '700mb'" "--maxmemory-policy volatile-lru" --proc-title-template --my--title--template --loglevel verbose
      ```
      
      ## Changes around --save
      Also in this PR, we undo the breaking change we made in #10660 on purpose.
      1. `redis-server redis.conf --save --loglevel verbose` (missing `save` argument before anotehr argument).
          In 7.0.1, it was throwing an wrong arg error.
          Now it will work and reset the save, similar to how it used to be in 7.0.0 and 6.2.x.
      3. `redis-server redis.conf --loglevel verbose --save` (missing `save` argument as last argument).
          In 6.2, it did not reset the save, which was a bug (inconsistent with the previous bullet).
          Now we will make it work and reset the save as well (a bug fix).
      d443e312
    • Viktor Söderqvist's avatar
      Add RM_SetClientNameById and RM_GetClientNameById (#10839) · 6272ca60
      Viktor Söderqvist authored
      Adding Module APIs to let the module read and set the client name of an arbitrary connection.
      6272ca60
    • Binbin's avatar
      Sync argv_len var back after command call in execCommand (#10900) · d96cf363
      Binbin authored
      This is harmless, we only restore mstate to make sure we
      free the right pointer in freeClientMultiState, but it'll
      be nicer to also sync that argv_len var back.
      d96cf363
    • Elvina Yakubova's avatar
      Replace regular array to array of structs to get rid of false sharing (#10892) · e2cf3867
      Elvina Yakubova authored
      There are a lot of false sharing cache misses in line 4013 inside getIOPendingCount function.
      The reason is that elements of io_threads_pending array access the same cache line from different
      threads, so it is better to represent it as an array of structures with fields aligned to cache line size.
      This change should improve performance (in particular, it affects the latency metric, we saw up to 3% improvement).
      e2cf3867
  11. 25 Jun, 2022 1 commit
    • Steve Lorello's avatar
      changing min,max in ZRANGE -> start stop (#10097) · a64b2948
      Steve Lorello authored
      In 6.2.0 with the introduction of the REV subcommand in ZRANGE, there was a semantic shift in the arguments of ZRANGE when the REV sub-command is executed. Without the sub-command `min` and `max` (the old names of the arguments) are appropriate because if you put the min value and the max value in everything works fine.
      
      ```bash
      127.0.0.1:6379> ZADD myset 0 foo
      (integer) 1
      127.0.0.1:6379> ZADD myset 1 bar
      (integer) 1
      127.0.0.1:6379> ZRANGE myset 0 inf BYSCORE
      1) "foo"
      2) "bar"
      ``` 
      
      However - if you add the `REV` subcommand, ordering the arguments `min` `max` breaks the command:
      
      ```bash
      127.0.0.1:6379> ZRANGE myset 0 inf BYSCORE REV
      (empty array)
      ```
      
      why? because `ZRANGE` with the `REV` sub-command is expecting the `max` first and the `min` second (because it's a reverse range like `ZREVRANGEBYSCORE`):
      
      ```bash
      127.0.0.1:6379> ZRANGE myset 0 inf BYSCORE REV
      (empty array)
      ```
      a64b2948
  12. 23 Jun, 2022 5 commits
  13. 22 Jun, 2022 1 commit
    • judeng's avatar
      fix redis-benchmark's bug: check if clients are created successfully in idle mode (#10891) · 49876158
      judeng authored
      my maxclients config:
      ```
      redis-cli config get maxclients
      1) "maxclients"
      2) "4064"
      ```
      
      Before this bug was fixed, creating 4065 clients appeared to be successful, but only 4064 were actually created```
      ```
      ./redis-benchmark -c 4065 -I
      Creating 4065 idle connections and waiting forever (Ctrl+C when done)
      cients: 4065
      ```
      
      now :
      ```
      ./redis-benchmark -c 4065 -I
      Creating 4065 idle connections and waiting forever (Ctrl+C when done)
      Error from server: ERR max number of clients reached
      
      ./redis-benchmark -c 4064 -I
      Creating 4064 idle connections and waiting forever (Ctrl+C when done)
      clients: 4064
      
      ```
      49876158
  14. 21 Jun, 2022 5 commits
    • Viktor Söderqvist's avatar
      Module API docs corrections (#10890) · 02acb8fd
      Viktor Söderqvist authored
      * Fix typo `RedisModule_CreatString` -> `RedisModule_CreateString` (multiple occurrences)
      * Make the markdown gen script change all `RM_` to `RedisModule_` even in code examples, etc.
      02acb8fd
    • Meir Shpilraien (Spielrein)'s avatar
      Fix crash on RM_Call with script mode. (#10886) · 61baabd8
      Meir Shpilraien (Spielrein) authored
      The PR fixes 2 issues:
      
      ### RM_Call crash on script mode
      
      `RM_Call` can potentially be called from a background thread where `server.current_client`
      are not set. In such case we get a crash on `NULL` dereference.
      The fix is to check first if `server.current_client` is `NULL`, if it does we should
      verify disc errors and readonly replica as we do to any normal clients (no masters nor AOF).
      
      ### RM_Call block OOM commands when not needed
      
      Again `RM_Call` can be executed on a background thread using a `ThreadSafeCtx`.
      In such case `server.pre_command_oom_state` can be irrelevant and should not be
      considered when check OOM state. This cause OOM commands to be blocked when
      not necessarily needed.
      
      In such case, check the actual used memory (and not the cached value). Notice that in
      order to know if the cached value can be used, we check that the ctx that was used on
      the `RM_Call` is a ThreadSafeCtx. Module writer can potentially abuse the API and use
      ThreadSafeCtx on the main thread. We consider this as a API miss used.
      61baabd8
    • Vlad's avatar
      Outdated comments, replace COUNTER_INIT_VAL with LFU_INIT_VAL, fix typo (#10888) · a3fdc9cd
      Vlad authored
      `COUNTER_INIT_VAL` doesn't exist in the code anymore so we can replace it with `LFU_INIT_VAL` entries
      a3fdc9cd
    • Bar Shaul's avatar
      Set replicas' configEpoch to 0 when loaded from cluster configuration file (#10798) · 091701f3
      Bar Shaul authored
      * Changed clusterLoadConfig to set the config epoch of replica nodes to 0 when loaded.
      091701f3
    • judeng's avatar
      Optimize the performance of clusterSendPing for large clusters (#10624) · ff641965
      judeng authored
      Optimize the performance of clusterSendPing by improving speed of checking for duplicate items in gossip.
      ff641965
  15. 20 Jun, 2022 1 commit
    • Tian's avatar
      Fsync directory while persisting AOF manifest, RDB file, and config file (#10737) · 99a425d0
      Tian authored
      The current process to persist files is `write` the data, `fsync` and `rename` the file,
      but a underlying problem is that the rename may be lost when a sudden crash like
      power outage and the directory hasn't been persisted.
      
      The article [Ensuring data reaches disk](https://lwn.net/Articles/457667/) mentions
      a safe way to update file should be:
      
      1. create a new temp file (on the same file system!)
      2. write data to the temp file
      3. fsync() the temp file
      4. rename the temp file to the appropriate name
      5. fsync() the containing directory
      
      This commit handles CONFIG REWRITE, AOF manifest, and RDB file (both for persistence,
      and the one the replica gets from the master).
      It doesn't handle (yet), ACL SAVE and Cluster configs, since these don't yet follow this pattern.
      99a425d0
  16. 19 Jun, 2022 3 commits
  17. 15 Jun, 2022 1 commit