1. 14 Jun, 2022 4 commits
    • Oran Agra's avatar
      Script that made modification will not break with unexpected NOREPLICAS error (#10855) · 8ef4f1db
      Oran Agra authored
      If a script made a modification and then was interrupted for taking too long.
      there's a chance redis will detect that a replica dropped and would like to reject
      write commands with NOREPLICAS due to insufficient good replicas.
      returning an error on a command in this case breaks the script atomicity.
      
      The same could in theory happen with READONLY, MISCONF, but i don't think
      these state changes can happen during script execution.
      8ef4f1db
    • Oran Agra's avatar
      Allow ECHO in loading and stale modes (#10853) · ffa00770
      Oran Agra authored
      I noticed that scripting.tcl uses INFO from within a script and thought it's an
      overkill and concluded it's nicer to use another CMD_STALE command,
      decided to use ECHO, and then noticed it's not at all allowed in stale mode.
      probably overlooked at #6843
      ffa00770
    • hdyztmdqd's avatar
      Fix 2 comments in dict.c & redis-cli.c (#10860) · c65e5087
      hdyztmdqd authored
      * fix comment in dict.c
      
      fix comment in dict.c
      
      * Fix comment in redis-cli.c
      
      Fix comment in redis-cli.c
      c65e5087
    • Huang Zhw's avatar
      Throw -TRYAGAIN instead of -ASK on migrating nodes for multi-key commands when... · 78960ad5
      Huang Zhw authored
      Throw -TRYAGAIN instead of -ASK on migrating nodes for multi-key commands when the node only has some of the keys (#9526)
      
      * In cluster getNodeByQuery when target slot is in migrating state and
      the slot lack some keys but have at least one key, should return TRYAGAIN.
      
      Before this commit, when a node is in migrating state and recevies
      multiple keys command, if some keys don't exist, the command emits
      an `ASK` redirection.
      
      After this commit, if some keys exist and some keys don't exist, the
      command emits a TRYAGAIN error. If all keys don't exist, the command
      emits an `ASK` redirection.
      78960ad5
  2. 13 Jun, 2022 2 commits
    • Binbin's avatar
      redis-check-rdb add when_opcode check for module aux (#10859) · 0a2f7883
      Binbin authored
      In #9199, we add a `goto eoferr` in when_opcode check,
      this means that if the when_opcode check fails, we will
      abort the rdb loading, but this not reflected in the
      rdb-check tool. So someone can modify when_opcode to make
      rdb load fail, but rdb-check will report OK. Just a cleanup
      or a code consistency issue.
      ```
      serverLog: # Internal error in RDB reading offset 0, function at rdb.c:3055 -> bad when_opcode
      [offset 0] Checking RDB file dump.rdb
      [offset 109] \o/ RDB looks OK! \o/
      ```
      
      Plus a minor memory leak fix like #9860, note that it will
      exit immediately after the eoferr, so it is not strictly a
      leak, so it is also a small cleanup.
      0a2f7883
    • XiongDa's avatar
      Fix typo in replication.c (#10854) · d4595dd9
      XiongDa authored
      d4595dd9
  3. 12 Jun, 2022 3 commits
    • Johannes Truschnigg's avatar
      Correctly check for vm.overcommit_memory == 0 (#10841) · 9f3b4100
      Johannes Truschnigg authored
      A regression caused by #10636 (released in 7.0.1) causes Redis startup warning about overcommit to be missing when needed and printed when not. 
      
      Also, using atoi() to convert the string's value to an integer cannot discern
      between an actual 0 (zero) having been read, or a conversion error. 
      9f3b4100
    • XiongDa's avatar
      Fix 3 comments in server.c (#10844) · abb2ea7e
      XiongDa authored
      abb2ea7e
    • Binbin's avatar
      Fixed SET and BITFIELD commands being wrongly marked movablekeys (#10837) · 92fb4f4f
      Binbin authored
      
      
      The SET and BITFIELD command were added `get_keys_function` in #10148, causing
      them to be wrongly marked movablekeys in `populateCommandMovableKeys`.
      
      This was an unintended side effect introduced in #10148 (7.0 RC1)
      which could cause some clients an extra round trip for these commands in cluster mode.
      
      Since we define movablekeys as a way to determine if the legacy range [first, last, step]
      doesn't find all keys, then we need a completely different approach.
      
      The right approach should be to check if the legacy range covers all key-specs,
      and if none of the key-specs have the INCOMPLETE flag. 
      This way, we don't need to look at getkeys_proc of VARIABLE_FLAG at all.
      Probably with the exception of modules, who may still not be using key-specs.
      
      In this PR, we removed `populateCommandMovableKeys` and put its logic in
      `populateCommandLegacyRangeSpec`.
      In order to properly serve both old and new modules, we must probably keep relying
      CMD_MODULE_GETKEYS, but do that only for modules that don't declare key-specs. 
      For ones that do, we need to take the same approach we take with native redis commands.
      
      This approach was proposed by Oran. Fixes #10833
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      92fb4f4f
  4. 11 Jun, 2022 1 commit
    • Binbin's avatar
      Fix crash when overcommit_memory is inaccessible (#10848) · 62ac1ab0
      Binbin authored
      When `/proc/sys/vm/overcommit_memory` is inaccessible, fp is NULL.
      `checkOvercommit` will return -1 without setting err_msg, and then
      the err_msg is used to print the log, crash the server.
      Set the err_msg variables to Null when declaring it, seems safer.
      
      And the overcommit_memory error log will print two "WARNING",
      like `WARNING WARNING overcommit_memory is set to 0!`, this PR
      also removes the second WARNING in `checkOvercommit`.
      
      Reported in #10846. Fixes #10846. Introduced in #10636 (7.0.1)
      62ac1ab0
  5. 09 Jun, 2022 2 commits
  6. 08 Jun, 2022 1 commit
  7. 07 Jun, 2022 6 commits
    • Petr Vaněk's avatar
      Update musl libc detection pattern (#10826) · f22bfe86
      Petr Vaněk authored
      This change fixes failing `integration/logging.tcl` test in Gentoo with
      musl libc, where `ldd` returns
      ```
      libc.so => /lib/ld-musl-x86_64.so.1 (0x7f9d5f171000)
      ```
      unlike Alpine's
      ```
      libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7f82cfa16000)
      ```
      The solution is to extend matching pattern introduced in #8532.
      f22bfe86
    • Binbin's avatar
      Increment the stat_rdb_saves counter in SAVE command (#10827) · ae9764ea
      Binbin authored
      Currently, we only increment stat_rdb_saves in rdbSaveBackground,
      we should also increment it in the SAVE command.
      
      We concluded there's no need to increment when:
      1. saving a base file for an AOF
      2. when saving an empty rdb file to delete an old one
      3. when saving to sockets (not creating a persistence / snapshot file)
      
      The stat counter was introduced in #10178
      
      * fix a wrong comment in startSaving
      ae9764ea
    • ls-2018's avatar
      b8665b87
    • Bjorn Svensson's avatar
      Documentation fixes of `BITFIELD_RO` and `XINFO STREAM` (#10823) · 1067cfb3
      Bjorn Svensson authored
      Correcting the introduction version of the command `BITFIELD_RO`
      Command added by commit: b3e4abf0 
      
      Add history info of the `FULL` modifier in `XINFO STREAM`
      Modifier was added by commit: 1e2aee39
      
      (Includes output from `./utils/generate-command-code.py`)
      1067cfb3
    • Binbin's avatar
      Handle multiple_token flag in generate-command-help.rb (#10822) · 3d56f607
      Binbin authored
      Currently generate-command.help.rb dose not handle the
      multiple_token flag, handle this flag in this PR.
      The format is the same as redis-cli rendering.
      ```diff
      - bitfield_ro key GET encoding offset [encoding offset ...]
      + bitfield_ro key GET encoding offset [GET encoding offset ...]
      ```
      
      Re run generate-command-code.py which was forget in #10820.
      Also change the flag value from string to bool, like "true" to true
      3d56f607
    • ranshid's avatar
      Fix bitfield_ro documentation (#10820) · b4772f63
      ranshid authored
      Current documentation only states a single GET token is needed which is not true. Marking the command with multiple_token.
      b4772f63
  8. 06 Jun, 2022 4 commits
    • Oran Agra's avatar
      update help.h (#10818) · b0272252
      Oran Agra authored
      before releasing 7.0.1
      b0272252
    • Oran Agra's avatar
      crash report instructions (#10816) · 475563e2
      Oran Agra authored
      Trying to avoid people opening crash report issues about module crashes and ARM QEMU bugs.
      475563e2
    • DarrenJiang13's avatar
      Split instantaneous_repl_total_kbps to instantaneous_input_repl_kbps and... · f5585834
      DarrenJiang13 authored
      Split instantaneous_repl_total_kbps to instantaneous_input_repl_kbps and instantaneous_output_repl_kbps. (#10810)
      
      A supplement to https://github.com/redis/redis/pull/10062
      Split `instantaneous_repl_total_kbps` to `instantaneous_input_repl_kbps` and `instantaneous_output_repl_kbps`. 
      ## Work:
      This PR:
      - delete 1 info field:
          - `instantaneous_repl_total_kbps`
      - add 2 info fields:
          - `instantaneous_input_repl_kbps / instantaneous_output_repl_kbps`
      ## Result:
      - master
      ```
      total_net_input_bytes:26633673
      total_net_output_bytes:21716596
      total_net_repl_input_bytes:0
      total_net_repl_output_bytes:18433052
      instantaneous_input_kbps:0.02
      instantaneous_output_kbps:0.00
      instantaneous_input_repl_kbps:0.00
      instantaneous_output_repl_kbps:0.00
      ```
      - slave
      ```
      total_net_input_bytes:18433212
      total_net_output_bytes:94790
      total_net_repl_input_bytes:18433052
      total_net_repl_output_bytes:0
      instantaneous_input_kbps:0.00
      instantaneous_output_kbps:0.05
      instantaneous_input_repl_kbps:0.00
      instantaneous_output_repl_kbps:0.00
      ```
      f5585834
    • Mixficsol's avatar
      Update cluster.c (#10773) · c751d8a6
      Mixficsol authored
      On line 4068, redis has a logical nodeIsSlave(myself) on the outer if layer,
      which you can delete without having to repeat the decision
      c751d8a6
  9. 03 Jun, 2022 2 commits
  10. 02 Jun, 2022 3 commits
    • zhaozhao.zz's avatar
      rewrite alias config to original name (#10811) · a18c91d6
      zhaozhao.zz authored
      
      
      Redis 7 adds some new alias config like `hash-max-listpack-entries` alias `hash-max-ziplist-entries`.
      
      If a config file contains both real name and alias like this:
      ```
      hash-max-listpack-entries 20
      hash-max-ziplist-entries 20
      ```
      
      after set `hash-max-listpack-entries` to 100 and `config rewrite`, the config file becomes to:
      ```
      hash-max-listpack-entries 100
      hash-max-ziplist-entries 20
      ```
      
      we can see that the alias config is not modified, and users will get wrong config after restart.
      
      6.0 and 6.2 doesn't have this bug, since they only have the `slave` word alias.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      a18c91d6
    • chenyang8094's avatar
      Only print final log when aof is loaded successfully (#10808) · aae0ec25
      chenyang8094 authored
      Skip the print on AOF_NOT_EXIST status.
      aae0ec25
    • zhugezy's avatar
      Fix bugs in CONFIG REWRITE, omitting rename-command and include lines, and... · cf3323db
      zhugezy authored
      
      Fix bugs in CONFIG REWRITE, omitting rename-command and include lines, and inserting comments around module and acl configs (#10761)
      
      A regression from #10285 (redis 7.0).
      CONFIG REWRITE would put lines with: `include`, `rename-command`,
      `user`,  `loadmodule`, and any module specific config in a comment.
      
      For ACL `user`, `loadmodule` and module specific configs would be
      re-inserted at the end (instead of updating existing lines), so the only
      implication is a messy config file full of comments.
      
      But for `rename-command` and `include`, the implication would be that
      they're now missing, so a server restart would lose them.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      cf3323db
  11. 01 Jun, 2022 4 commits
    • Ozan Tezcan's avatar
      Fix Lua compile warning (#10805) · c81b5e55
      Ozan Tezcan authored
      Apparently, GCC 11.2.0 has a new fancy warning for misleading indentations.
      It prints a warning when BRET(b) is on the same line as the loop.
      c81b5e55
    • Oran Agra's avatar
      Expose script flags to processCommand for better handling (#10744) · df558618
      Oran Agra authored
      The important part is that read-only scripts (not just EVAL_RO
      and FCALL_RO, but also ones with `no-writes` executed by normal EVAL or
      FCALL), will now be permitted to run during CLIENT PAUSE WRITE (unlike
      before where only the _RO commands would be processed).
      
      Other than that, some errors like OOM, READONLY, MASTERDOWN are now
      handled by processCommand, rather than the command itself affects the
      error string (and even error code in some cases), and command stats.
      
      Besides that, now the `may-replicate` commands, PFCOUNT and PUBLISH, will
      be considered `write` commands in scripts and will be blocked in all
      read-only scripts just like other write commands.
      They'll also be blocked in EVAL_RO (i.e. even for scripts without the
      `no-writes` shebang flag.
      
      This commit also hides the `may_replicate` flag from the COMMAND command
      output. this is a **breaking change**.
      
      background about may_replicate:
      We don't want to expose a no-may-replicate flag or alike to scripts, since we
      consider the may-replicate thing an internal concern of redis, that we may
      some day get rid of.
      In fact, the may-replicate flag was initially introduced to flag EVAL: since
      we didn't know what it's gonna do ahead of execution, before function-flags
      existed). PUBLISH and PFCOUNT, both of which because they have side effects
      which may some day be fixed differently.
      
      code changes:
      The changes in eval.c are mostly code re-ordering:
      - evalCalcFunctionName is extracted out of evalGenericCommand
      - evalExtractShebangFlags is extracted luaCreateFunction
      - evalGetCommandFlags is new code
      df558618
    • Oran Agra's avatar
      Fix broken protocol in MISCONF error, RM_Yield bugs, RM_Call(EVAL) OOM check... · b2061de2
      Oran Agra authored
      Fix broken protocol in MISCONF error, RM_Yield bugs, RM_Call(EVAL) OOM check bug, and new RM_Call checks. (#10786)
      
      * Fix broken protocol when redis can't persist to RDB (general commands, not
        modules), excessive newline. regression of #10372 (7.0 RC3)
      * Fix broken protocol when Redis can't persist to AOF (modules and
        scripts), missing newline.
      * Fix bug in OOM check of EVAL scripts called from RM_Call.
        set the cached OOM state for scripts before executing module commands too,
        so that it can serve scripts that are executed by modules.
        i.e. in the past EVAL executed by RM_Call could have either falsely
        fail or falsely succeeded because of a wrong cached OOM state flag.
      * Fix bugs with RM_Yield:
        1. SHUTDOWN should only accept the NOSAVE mode
        2. Avoid eviction during yield command processing.
        3. Avoid processing master client commands while yielding from another client
      * Add new two more checks to RM_Call script mode.
        1. READONLY You can't write against a read only replica
        2. MASTERDOWN Link with MASTER is down and `replica-serve-stale-data` is set to `no`
      * Add new RM_Call flag to let redis automatically refuse `deny-oom` commands
        while over the memory limit. 
      * Add tests to cover various errors from Scripts, Modules, Modules
        calling scripts, and Modules calling commands in script mode.
      
      Add tests:
      * Looks like the MISCONF error was completely uncovered by the tests,
        add tests for it, including from scripts, and modules
      * Add tests for NOREPLICAS from scripts
      * Add tests for the various errors in module RM_Call, including RM_Call that
        calls EVAL, and RM_call in "eval mode". that includes:
        NOREPLICAS, READONLY, MASTERDOWN, MISCONF
      b2061de2
    • filipe oliveira's avatar
      Moving client flags to a more cache friendly position within client struct (#10697) · 6a6e911f
      filipe oliveira authored
      Move the client flags to a more cache friendly position within the client struct
      we regain the lost 2% of CPU cycles since v6.2 ( from 630532.57 to 647449.80 ops/sec ).
      These are due to higher rate of calls to getClientType due to changes in #9166 and #10020 
      6a6e911f
  12. 31 May, 2022 5 commits
    • menwen's avatar
      Add module API RM_MallocUsableSize (#10795) · 42fbf064
      menwen authored
      This allows the module to know the usable size of an allocation
      it made, rather than the consumed size.
      42fbf064
    • zhaozhao.zz's avatar
      rename channel to shardchannel in sharded pubsub commands (#10738) · 805191c7
      zhaozhao.zz authored
      since the sharded pubsub is different with pubsub, it's better to give users a hint to make it more clear.
      805191c7
    • DarrenJiang13's avatar
      Adds isolated netstats for replication. (#10062) · bb1de082
      DarrenJiang13 authored
      
      
      The amount of `server.stat_net_output_bytes/server.stat_net_input_bytes`
      is actually the sum of replication flow and users' data flow. 
      It may cause confusions like this:
      "Why does my server get such a large output_bytes while I am doing nothing? ". 
      
      After discussions and revisions, now here is the change about what this
      PR brings (final version before merge):
      - 2 server variables to count the network bytes during replication,
           including fullsync and propagate bytes.
           - `server.stat_net_repl_output_bytes`/`server.stat_net_repl_input_bytes`
      - 3 info fields to print the input and output of repl bytes and instantaneous
           value of total repl bytes.
           - `total_net_repl_input_bytes` / `total_net_repl_output_bytes`
           - `instantaneous_repl_total_kbps`
      - 1 new API `rioCheckType()` to check the type of rio. So we can use this
           to distinguish between diskless and diskbased replication
      - 2 new counting items to keep network statistics consistent between master
           and slave
          - rdb portion during diskless replica. in `rdbLoadProgressCallback()`
          - first line of the full sync payload. in `readSyncBulkPayload()`
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      bb1de082
    • Harkrishn Patro's avatar
      Sharded pubsub publish messagebulk as smessage (#10792) · 4065b4f2
      Harkrishn Patro authored
      To easily distinguish between sharded channel message and a global
      channel message, introducing `smessage` (instead of `message`) as
      message bulk for sharded channel publish message.
      
      This is gonna be a breaking change in 7.0.1!
      
      Background:
      Sharded pubsub introduced in redis 7.0, but after the release we quickly
      realized that the fact that it's problematic that the client can't distinguish
      between normal (global) pubsub messages and sharded ones.
      This is important because the same connection can subscribe to both,
      but messages sent to one pubsub system are not propagated to the
      other (they're completely separate), so if one connection is used to
      subscribe to both, we need to assist the client library to know which
      message it got so it can forward it to the correct callback.
      4065b4f2
    • Wen Hui's avatar
      Fix streamParseAddOrTrimArgsOrReply function minor comment issue (#10783) · d7ae8587
      Wen Hui authored
      
      
      When I read the source codes, I have no idea where the option "age" come from.
      Co-authored-by: default avatarUbuntu <lucas.guang.yang1@huawei.com>
      Co-authored-by: default avatarguybe7 <guy.benoish@redislabs.com>
      d7ae8587
  13. 30 May, 2022 1 commit
  14. 29 May, 2022 2 commits