1. 27 Jun, 2022 1 commit
    • Viktor Söderqvist's avatar
      Add missing REDISMODULE_CLIENTINFO_INITIALIZER (#10885) · 6af02100
      Viktor Söderqvist authored
      The module API docs mentions this macro, but it was not defined (so no one could have used it).
      
      Instead of adding it as is, we decided to add a _V1 macro, so that if / when we some day extend this struct,
      modules that use this API and don't need the extra fields, will still use the old version
      and still be compatible with older redis version (despite being compiled with newer redismodule.h)
      6af02100
  2. 26 Jun, 2022 5 commits
    • RinChanNOW!'s avatar
      Support conversion between `RedisModuleString` and `unsigned long long` (#10889) · 28546373
      RinChanNOW! authored
      
      
      Since the ranges of `unsigned long long` and `long long` are different, we cannot read an
      `unsigned long long` integer from a `RedisModuleString` by `RedisModule_StringToLongLong` . 
      
      So I added two new Redis Module APIs to support the conversion between these two types:
      * `RedisModule_StringToULongLong`
      * `RedisModule_CreateStringFromULongLong`
      Signed-off-by: default avatarRinChanNOWWW <hzy427@gmail.com>
      28546373
    • Binbin's avatar
      redis-server command line arguments allow passing config name and value in the same arg (#10866) · d443e312
      Binbin authored
      This commit has two topics.
      
      ## Passing config name and value in the same arg
      In #10660 (Redis 7.0.1), when we supported the config values that can start with `--` prefix (one of the two topics of that PR),
      we broke another pattern: `redis-server redis.config "name value"`, passing both config name
      and it's value in the same arg, see #10865
      
      This wasn't a intended change (i.e we didn't realize this pattern used to work).
      Although this is a wrong usage, we still like to fix it.
      
      Now we support something like:
      ```
      src/redis-server redis.conf "--maxmemory '700mb'" "--maxmemory-policy volatile-lru" --proc-title-template --my--title--template --loglevel verbose
      ```
      
      ## Changes around --save
      Also in this PR, we undo the breaking change we made in #10660 on purpose.
      1. `redis-server redis.conf --save --loglevel verbose` (missing `save` argument before anotehr argument).
          In 7.0.1, it was throwing an wrong arg error.
          Now it will work and reset the save, similar to how it used to be in 7.0.0 and 6.2.x.
      3. `redis-server redis.conf --loglevel verbose --save` (missing `save` argument as last argument).
          In 6.2, it did not reset the save, which was a bug (inconsistent with the previous bullet).
          Now we will make it work and reset the save as well (a bug fix).
      d443e312
    • Viktor Söderqvist's avatar
      Add RM_SetClientNameById and RM_GetClientNameById (#10839) · 6272ca60
      Viktor Söderqvist authored
      Adding Module APIs to let the module read and set the client name of an arbitrary connection.
      6272ca60
    • Binbin's avatar
      Sync argv_len var back after command call in execCommand (#10900) · d96cf363
      Binbin authored
      This is harmless, we only restore mstate to make sure we
      free the right pointer in freeClientMultiState, but it'll
      be nicer to also sync that argv_len var back.
      d96cf363
    • Elvina Yakubova's avatar
      Replace regular array to array of structs to get rid of false sharing (#10892) · e2cf3867
      Elvina Yakubova authored
      There are a lot of false sharing cache misses in line 4013 inside getIOPendingCount function.
      The reason is that elements of io_threads_pending array access the same cache line from different
      threads, so it is better to represent it as an array of structures with fields aligned to cache line size.
      This change should improve performance (in particular, it affects the latency metric, we saw up to 3% improvement).
      e2cf3867
  3. 25 Jun, 2022 1 commit
    • Steve Lorello's avatar
      changing min,max in ZRANGE -> start stop (#10097) · a64b2948
      Steve Lorello authored
      In 6.2.0 with the introduction of the REV subcommand in ZRANGE, there was a semantic shift in the arguments of ZRANGE when the REV sub-command is executed. Without the sub-command `min` and `max` (the old names of the arguments) are appropriate because if you put the min value and the max value in everything works fine.
      
      ```bash
      127.0.0.1:6379> ZADD myset 0 foo
      (integer) 1
      127.0.0.1:6379> ZADD myset 1 bar
      (integer) 1
      127.0.0.1:6379> ZRANGE myset 0 inf BYSCORE
      1) "foo"
      2) "bar"
      ``` 
      
      However - if you add the `REV` subcommand, ordering the arguments `min` `max` breaks the command:
      
      ```bash
      127.0.0.1:6379> ZRANGE myset 0 inf BYSCORE REV
      (empty array)
      ```
      
      why? because `ZRANGE` with the `REV` sub-command is expecting the `max` first and the `min` second (because it's a reverse range like `ZREVRANGEBYSCORE`):
      
      ```bash
      127.0.0.1:6379> ZRANGE myset 0 inf BYSCORE REV
      (empty array)
      ```
      a64b2948
  4. 23 Jun, 2022 5 commits
  5. 22 Jun, 2022 1 commit
    • judeng's avatar
      fix redis-benchmark's bug: check if clients are created successfully in idle mode (#10891) · 49876158
      judeng authored
      my maxclients config:
      ```
      redis-cli config get maxclients
      1) "maxclients"
      2) "4064"
      ```
      
      Before this bug was fixed, creating 4065 clients appeared to be successful, but only 4064 were actually created```
      ```
      ./redis-benchmark -c 4065 -I
      Creating 4065 idle connections and waiting forever (Ctrl+C when done)
      cients: 4065
      ```
      
      now :
      ```
      ./redis-benchmark -c 4065 -I
      Creating 4065 idle connections and waiting forever (Ctrl+C when done)
      Error from server: ERR max number of clients reached
      
      ./redis-benchmark -c 4064 -I
      Creating 4064 idle connections and waiting forever (Ctrl+C when done)
      clients: 4064
      
      ```
      49876158
  6. 21 Jun, 2022 5 commits
    • Viktor Söderqvist's avatar
      Module API docs corrections (#10890) · 02acb8fd
      Viktor Söderqvist authored
      * Fix typo `RedisModule_CreatString` -> `RedisModule_CreateString` (multiple occurrences)
      * Make the markdown gen script change all `RM_` to `RedisModule_` even in code examples, etc.
      02acb8fd
    • Meir Shpilraien (Spielrein)'s avatar
      Fix crash on RM_Call with script mode. (#10886) · 61baabd8
      Meir Shpilraien (Spielrein) authored
      The PR fixes 2 issues:
      
      ### RM_Call crash on script mode
      
      `RM_Call` can potentially be called from a background thread where `server.current_client`
      are not set. In such case we get a crash on `NULL` dereference.
      The fix is to check first if `server.current_client` is `NULL`, if it does we should
      verify disc errors and readonly replica as we do to any normal clients (no masters nor AOF).
      
      ### RM_Call block OOM commands when not needed
      
      Again `RM_Call` can be executed on a background thread using a `ThreadSafeCtx`.
      In such case `server.pre_command_oom_state` can be irrelevant and should not be
      considered when check OOM state. This cause OOM commands to be blocked when
      not necessarily needed.
      
      In such case, check the actual used memory (and not the cached value). Notice that in
      order to know if the cached value can be used, we check that the ctx that was used on
      the `RM_Call` is a ThreadSafeCtx. Module writer can potentially abuse the API and use
      ThreadSafeCtx on the main thread. We consider this as a API miss used.
      61baabd8
    • Vlad's avatar
      Outdated comments, replace COUNTER_INIT_VAL with LFU_INIT_VAL, fix typo (#10888) · a3fdc9cd
      Vlad authored
      `COUNTER_INIT_VAL` doesn't exist in the code anymore so we can replace it with `LFU_INIT_VAL` entries
      a3fdc9cd
    • Bar Shaul's avatar
      Set replicas' configEpoch to 0 when loaded from cluster configuration file (#10798) · 091701f3
      Bar Shaul authored
      * Changed clusterLoadConfig to set the config epoch of replica nodes to 0 when loaded.
      091701f3
    • judeng's avatar
      Optimize the performance of clusterSendPing for large clusters (#10624) · ff641965
      judeng authored
      Optimize the performance of clusterSendPing by improving speed of checking for duplicate items in gossip.
      ff641965
  7. 20 Jun, 2022 1 commit
    • Tian's avatar
      Fsync directory while persisting AOF manifest, RDB file, and config file (#10737) · 99a425d0
      Tian authored
      The current process to persist files is `write` the data, `fsync` and `rename` the file,
      but a underlying problem is that the rename may be lost when a sudden crash like
      power outage and the directory hasn't been persisted.
      
      The article [Ensuring data reaches disk](https://lwn.net/Articles/457667/) mentions
      a safe way to update file should be:
      
      1. create a new temp file (on the same file system!)
      2. write data to the temp file
      3. fsync() the temp file
      4. rename the temp file to the appropriate name
      5. fsync() the containing directory
      
      This commit handles CONFIG REWRITE, AOF manifest, and RDB file (both for persistence,
      and the one the replica gets from the master).
      It doesn't handle (yet), ACL SAVE and Cluster configs, since these don't yet follow this pattern.
      99a425d0
  8. 19 Jun, 2022 3 commits
  9. 15 Jun, 2022 1 commit
  10. 14 Jun, 2022 5 commits
    • Oran Agra's avatar
      optimize zset conversion on large ZRANGESTORE (#10789) · 21891003
      Oran Agra authored
      when we know the size of the zset we're gonna store in advance,
      we can check if it's greater than the listpack encoding threshold,
      in which case we can create a skiplist from the get go, and avoid
      converting the listpack to skiplist later after it was already populated.
      21891003
    • Oran Agra's avatar
      Script that made modification will not break with unexpected NOREPLICAS error (#10855) · 8ef4f1db
      Oran Agra authored
      If a script made a modification and then was interrupted for taking too long.
      there's a chance redis will detect that a replica dropped and would like to reject
      write commands with NOREPLICAS due to insufficient good replicas.
      returning an error on a command in this case breaks the script atomicity.
      
      The same could in theory happen with READONLY, MISCONF, but i don't think
      these state changes can happen during script execution.
      8ef4f1db
    • Oran Agra's avatar
      Allow ECHO in loading and stale modes (#10853) · ffa00770
      Oran Agra authored
      I noticed that scripting.tcl uses INFO from within a script and thought it's an
      overkill and concluded it's nicer to use another CMD_STALE command,
      decided to use ECHO, and then noticed it's not at all allowed in stale mode.
      probably overlooked at #6843
      ffa00770
    • hdyztmdqd's avatar
      Fix 2 comments in dict.c & redis-cli.c (#10860) · c65e5087
      hdyztmdqd authored
      * fix comment in dict.c
      
      fix comment in dict.c
      
      * Fix comment in redis-cli.c
      
      Fix comment in redis-cli.c
      c65e5087
    • Huang Zhw's avatar
      Throw -TRYAGAIN instead of -ASK on migrating nodes for multi-key commands when... · 78960ad5
      Huang Zhw authored
      Throw -TRYAGAIN instead of -ASK on migrating nodes for multi-key commands when the node only has some of the keys (#9526)
      
      * In cluster getNodeByQuery when target slot is in migrating state and
      the slot lack some keys but have at least one key, should return TRYAGAIN.
      
      Before this commit, when a node is in migrating state and recevies
      multiple keys command, if some keys don't exist, the command emits
      an `ASK` redirection.
      
      After this commit, if some keys exist and some keys don't exist, the
      command emits a TRYAGAIN error. If all keys don't exist, the command
      emits an `ASK` redirection.
      78960ad5
  11. 13 Jun, 2022 2 commits
    • Binbin's avatar
      redis-check-rdb add when_opcode check for module aux (#10859) · 0a2f7883
      Binbin authored
      In #9199, we add a `goto eoferr` in when_opcode check,
      this means that if the when_opcode check fails, we will
      abort the rdb loading, but this not reflected in the
      rdb-check tool. So someone can modify when_opcode to make
      rdb load fail, but rdb-check will report OK. Just a cleanup
      or a code consistency issue.
      ```
      serverLog: # Internal error in RDB reading offset 0, function at rdb.c:3055 -> bad when_opcode
      [offset 0] Checking RDB file dump.rdb
      [offset 109] \o/ RDB looks OK! \o/
      ```
      
      Plus a minor memory leak fix like #9860, note that it will
      exit immediately after the eoferr, so it is not strictly a
      leak, so it is also a small cleanup.
      0a2f7883
    • XiongDa's avatar
      Fix typo in replication.c (#10854) · d4595dd9
      XiongDa authored
      d4595dd9
  12. 12 Jun, 2022 3 commits
    • Johannes Truschnigg's avatar
      Correctly check for vm.overcommit_memory == 0 (#10841) · 9f3b4100
      Johannes Truschnigg authored
      A regression caused by #10636 (released in 7.0.1) causes Redis startup warning about overcommit to be missing when needed and printed when not. 
      
      Also, using atoi() to convert the string's value to an integer cannot discern
      between an actual 0 (zero) having been read, or a conversion error. 
      9f3b4100
    • XiongDa's avatar
      Fix 3 comments in server.c (#10844) · abb2ea7e
      XiongDa authored
      abb2ea7e
    • Binbin's avatar
      Fixed SET and BITFIELD commands being wrongly marked movablekeys (#10837) · 92fb4f4f
      Binbin authored
      
      
      The SET and BITFIELD command were added `get_keys_function` in #10148, causing
      them to be wrongly marked movablekeys in `populateCommandMovableKeys`.
      
      This was an unintended side effect introduced in #10148 (7.0 RC1)
      which could cause some clients an extra round trip for these commands in cluster mode.
      
      Since we define movablekeys as a way to determine if the legacy range [first, last, step]
      doesn't find all keys, then we need a completely different approach.
      
      The right approach should be to check if the legacy range covers all key-specs,
      and if none of the key-specs have the INCOMPLETE flag. 
      This way, we don't need to look at getkeys_proc of VARIABLE_FLAG at all.
      Probably with the exception of modules, who may still not be using key-specs.
      
      In this PR, we removed `populateCommandMovableKeys` and put its logic in
      `populateCommandLegacyRangeSpec`.
      In order to properly serve both old and new modules, we must probably keep relying
      CMD_MODULE_GETKEYS, but do that only for modules that don't declare key-specs. 
      For ones that do, we need to take the same approach we take with native redis commands.
      
      This approach was proposed by Oran. Fixes #10833
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      92fb4f4f
  13. 11 Jun, 2022 1 commit
    • Binbin's avatar
      Fix crash when overcommit_memory is inaccessible (#10848) · 62ac1ab0
      Binbin authored
      When `/proc/sys/vm/overcommit_memory` is inaccessible, fp is NULL.
      `checkOvercommit` will return -1 without setting err_msg, and then
      the err_msg is used to print the log, crash the server.
      Set the err_msg variables to Null when declaring it, seems safer.
      
      And the overcommit_memory error log will print two "WARNING",
      like `WARNING WARNING overcommit_memory is set to 0!`, this PR
      also removes the second WARNING in `checkOvercommit`.
      
      Reported in #10846. Fixes #10846. Introduced in #10636 (7.0.1)
      62ac1ab0
  14. 09 Jun, 2022 2 commits
  15. 08 Jun, 2022 1 commit
  16. 07 Jun, 2022 3 commits
    • Petr Vaněk's avatar
      Update musl libc detection pattern (#10826) · f22bfe86
      Petr Vaněk authored
      This change fixes failing `integration/logging.tcl` test in Gentoo with
      musl libc, where `ldd` returns
      ```
      libc.so => /lib/ld-musl-x86_64.so.1 (0x7f9d5f171000)
      ```
      unlike Alpine's
      ```
      libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7f82cfa16000)
      ```
      The solution is to extend matching pattern introduced in #8532.
      f22bfe86
    • Binbin's avatar
      Increment the stat_rdb_saves counter in SAVE command (#10827) · ae9764ea
      Binbin authored
      Currently, we only increment stat_rdb_saves in rdbSaveBackground,
      we should also increment it in the SAVE command.
      
      We concluded there's no need to increment when:
      1. saving a base file for an AOF
      2. when saving an empty rdb file to delete an old one
      3. when saving to sockets (not creating a persistence / snapshot file)
      
      The stat counter was introduced in #10178
      
      * fix a wrong comment in startSaving
      ae9764ea
    • ls-2018's avatar
      b8665b87