1. 01 Aug, 2022 1 commit
  2. 31 Jul, 2022 2 commits
  3. 28 Jul, 2022 1 commit
    • Binbin's avatar
      Avoid false positive out-of-bounds in writeForgottenNodePingExt (#11053) · 90f35cea
      Binbin authored
      In clusterMsgPingExtForgottenNode, sizeof(name) is CLUSTER_NAMELEN,
      and sizeof(clusterMsgPingExtForgottenNode) is > CLUSTER_NAMELEN.
      Doing a (name + sizeof(clusterMsgPingExtForgottenNode)) sanitizer
      generates an out-of-bounds error which is a false positive in here
      90f35cea
  4. 27 Jul, 2022 2 commits
    • Binbin's avatar
      Fix bgsaveerr issue in psync wrong offset test (#11043) · e7144693
      Binbin authored
      The kill above is sometimes successful and sometimes already too late.
      The PING in pysnc wrong offset test got rejected by bgsaveerr because
      lastbgsave_status is C_ERR.
      
      In theory, using diskless can avoid PING being affected, because when
      the replica is dropped, we will kill the child with SIGUSR1, and this
      will not affect lastbgsave_status.
      
      Anyway, this kill is not particularly needed here, dropping the kill
      is the best one, since we do have the waitForBgsave, so just let it
      take care of the bgsave. No need for fast termination.
      e7144693
    • guybe7's avatar
      Adds RM_Microseconds and RM_CachedMicroseconds (#11016) · 45c99d70
      guybe7 authored
      RM_Microseconds
      Return the wall-clock Unix time, in microseconds
      
      RM_CachedMicroseconds
      Returns a cached copy of the Unix time, in microseconds.
      It is updated in the server cron job and before executing a command.
      It is useful for complex call stacks, such as a command causing a
      key space notification, causing a module to execute a RedisModule_Call,
      causing another notification, etc.
      It makes sense that all these callbacks would use the same clock.
      45c99d70
  5. 26 Jul, 2022 4 commits
    • Binbin's avatar
      Change the return value of rdbLoad function to enums (#11039) · 00097bf4
      Binbin authored
      The reason we do this is because in #11036, we added error
      log message when failing to open RDB file for reading.
      In loadDdataFromDisk we call rdbLoad and also check errno,
      now the logging corrupts errno (reported in alpine daily).
      
      It is not safe to rely on errno as we do today, so we change
      the return value of rdbLoad function to enums, like we have
      when loading an AOF.
      00097bf4
    • Huang Zhw's avatar
      When client tracking is on, invalidation message of flushdb in a (#11038) · 6f0a27e3
      Huang Zhw authored
      When FLUSHDB / FLUSHALL / SWAPDB is inside MULTI / EXEC, the
      client side tracking invalidation message was interleaved with transaction response.
      6f0a27e3
    • Meir Shpilraien (Spielrein)'s avatar
      Fix #11030, use lua_rawget to avoid triggering metatables and crash. (#11032) · 020e046b
      Meir Shpilraien (Spielrein) authored
      Fix #11030, use lua_rawget to avoid triggering metatables.
      
      #11030 shows how return `_G` from the Lua script (either function or eval), cause the
      Lua interpreter to Panic and the Redis processes to exit with error code 1.
      Though return `_G` only panic on Redis 7 and 6.2.7, the underline issue exists on older
      versions as well (6.0 and 6.2). The underline issue is returning a table with a metatable
      such that the metatable raises an error.
      
      The following example demonstrate the issue:
      ```
      127.0.0.1:6379> eval "local a = {}; setmetatable(a,{__index=function() foo() end}) return a" 0
      Error: Server closed the connection
      ```
      ```
      PANIC: unprotected error in call to Lua API (user_script:1: Script attempted to access nonexistent global variable 'foo')
      ```
      
      The Lua panic happened because when returning the result to the client, Redis needs to
      introspect the returning table and transform the table into a resp. In order to scan the table,
      Redis uses `lua_gettable` api which might trigger the metatable (if exists) and might raise an error.
      This code is not running inside `pcall` (Lua protected call), so raising an error causes the
      Lua to panic and exit. Notice that this is not a crash, its a Lua panic that exit with error code 1.
      
      Returning `_G` panics on Redis 7 and 6.2.7 because on those versions `_G` has a metatable
      that raises error when trying to fetch a none existing key.
      
      ### Solution
      
      Instead of using `lua_gettable` that might raise error and cause the issue, use `lua_rawget`
      that simply return the value from the table without triggering any metatable logic.
      This is promised not to raise and error.
      
      The downside of this solution is that it might be considered as breaking change, if someone
      rely on metatable in the returned value. An alternative solution is to wrap this entire logic
      with `pcall` (Lua protected call), this alternative require a much bigger refactoring.
      
      ### Back Porting
      
      The same fix will work on older versions as well (6.2, 6.0). Notice that on those version,
      the issue can cause Redis to crash if inside the metatable logic there is an attempt to accesses
      Redis (`redis.call`). On 7.0, there is not crash and the `redis.call` is executed as if it was done
      from inside the script itself.
      
      ### Tests
      
      Tests was added the verify the fix
      020e046b
    • Viktor Söderqvist's avatar
      Gossip forgotten nodes on `CLUSTER FORGET` (#10869) · 5032de50
      Viktor Söderqvist authored
      Gossip the cluster node blacklist in ping and pong messages.
      This means that CLUSTER FORGET doesn't need to be sent to all nodes in a cluster.
      It can be sent to one or more nodes and then be propagated to the rest of them.
      
      For each blacklisted node, its node id and its remaining blacklist TTL is gossiped in a
      cluster bus ping extension (introduced in #9530).
      5032de50
  6. 25 Jul, 2022 2 commits
    • YaacovHazan's avatar
      Add error log message when failing to open RDB file for reading (#11036) · 33bd8fb9
      YaacovHazan authored
      When failing to open the rdb file, there was no specific error printed (unlike a
      corrupt file), so it was not clear what failed and why.
      33bd8fb9
    • Binbin's avatar
      fsync the old aof file when open a new INCR AOF (#11004) · 03fff10a
      Binbin authored
      
      
      In rewriteAppendOnlyFileBackground, after flushAppendOnlyFile(1),
      and before openNewIncrAofForAppend, we should call redis_fsync
      to fsync the aof file.
      
      Because we may open a new INCR AOF in openNewIncrAofForAppend,
      in the case of using everysec policy, the old AOF file may not
      be fsynced in time (or even at all).
      
      When using everysec, we don't want to pay the disk latency from
      the main thread, so we will do a background fsync.
      
      Adding a argument for bioCreateCloseJob, a `need_fsync` flag to
      indicate that a fsync is required before the file is closed. So we will
      fsync the old AOF file before we close it.
      
      A cleanup, we make union become a union, since the free_* args and
      the fd / fsync args are never used together.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      03fff10a
  7. 24 Jul, 2022 2 commits
  8. 20 Jul, 2022 3 commits
  9. 19 Jul, 2022 4 commits
    • Wen Hui's avatar
      Fix EVALSHA_RO and EVAL_RO command json file (#11015) · 56828bab
      Wen Hui authored
      
      
      these are missing from the RO_ commands, present in the other ones.
      Co-authored-by: default avatarUbuntu <lucas.guang.yang1@huawei.com>
      56828bab
    • Yossi Gottlieb's avatar
      CI: Update vmaction. (#11013) · b550a55c
      Yossi Gottlieb authored
      b550a55c
    • Binbin's avatar
      Set RM_StringCompare input args as const (#11010) · 95b88f67
      Binbin authored
      Following #10996, it forgot to modify RM_StringCompare in module.c
      
      Modified RM_StringCompare, compareStringObjectsWithFlags,
      compareStringObjects and collateStringObjects.
      95b88f67
    • Binbin's avatar
      Fix timing issue in cluster test (#11008) · 5ce64ab0
      Binbin authored
      A timing issue like this was reported in freebsd daily CI:
      ```
      *** [err]: Sanity test push cmd after resharding in tests/unit/cluster/cli.tcl
      Expected 'CLUSTERDOWN The cluster is down' to match '*MOVED*'
      ```
      
      We additionally wait for each node to reach a consensus on the cluster
      state in wait_for_condition to avoid the cluster down error.
      
      The fix just like #10495, quoting madolson's comment:
      Cluster check just verifies the the config state is self-consistent,
      waiting for cluster_state to be okay is an independent check that all
      the nodes actually believe each other are healthy.
      
      At the same time i noticed that unit/moduleapi/cluster.tcl has an exact
      same test, may have the same problem, also modified it.
      5ce64ab0
  10. 18 Jul, 2022 2 commits
    • Oran Agra's avatar
      Fix heap overflow corruption in XAUTOCLAIM (CVE-2022-31144) (#11002) · 2825b605
      Oran Agra authored
      The temporary array for deleted entries reply of XAUTOCLAIM was
      insufficient, but also in fact the COUNT argument should be used to
      control the size of the reply, so instead of terminating the loop by
      only counting the claimed entries, we'll count deleted entries as well.
      
      Fix #10968
      Addresses CVE-2022-31144
      2825b605
    • ranshid's avatar
      Avoid using unsafe C functions (#10932) · eacca729
      ranshid authored
      replace use of:
      sprintf --> snprintf
      strcpy/strncpy  --> redis_strlcpy
      strcat/strncat  --> redis_strlcat
      
      **why are we making this change?**
      Much of the code uses some unsafe variants or deprecated buffer handling
      functions.
      While most cases are probably not presenting any issue on the known path
      programming errors and unterminated strings might lead to potential
      buffer overflows which are not covered by tests.
      
      **As part of this PR we change**
      1. added implementation for redis_strlcpy and redis_strlcat based on the strl implementation: https://linux.die.net/man/3/strl
      2. change all occurrences of use of sprintf with use of snprintf
      3. change occurrences of use of  strcpy/strncpy with redis_strlcpy
      4. change occurrences of use of strcat/strncat with redis_strlcat
      5. change the behavior of ll2string/ull2string/ld2string so that it will always place null
        termination ('\0') on the output buffer in the first index. this was done in order to make
        the use of these functions more safe in cases were the user will not check the output
        returned by them (for example in rdbRemoveTempFile)
      6. we added a compiler directive to issue a deprecation error in case a use of
        sprintf/strcpy/strcat is found during compilation which will result in error during compile time.
        However keep in mind that since the deprecation attribute is not supported on all compilers,
        this is expected to fail during push workflows.
      
      
      **NOTE:** while this is only an initial milestone. We might also consider
      using the *_s implementation provided by the C11 Extensions (however not
      yet widly supported). I would also suggest to start
      looking at static code analyzers to track unsafe use cases.
      For example LLVM clang checker supports security.insecureAPI.DeprecatedOrUnsafeBufferHandling
      which can help locate unsafe function usage.
      https://clang.llvm.org/docs/analyzer/checkers.html#security-insecureapi-deprecatedorunsafebufferhandling-c
      The main reason not to onboard it at this stage is that the alternative
      excepted by clang is to use the C11 extensions which are not always
      supported by stdlib.
      eacca729
  11. 17 Jul, 2022 5 commits
  12. 13 Jul, 2022 2 commits
    • Oran Agra's avatar
      Avoid valgrind fishy value warning on corrupt restore payloads (#10937) · 599e59eb
      Oran Agra authored
      The corrupt dump fuzzer uncovered a valgrind warning saying:
      ```
      ==76370== Argument 'size' of function malloc has a fishy (possibly negative) value: -3744781444216323815
      ```
      This allocation would have failed (returning NULL) and being handled properly by redis (even before this change), but we also want to silence the valgrind warnings (which are checking that casting to ssize_t produces a non-negative value).
      
      The solution i opted for is to explicitly fail these allocations (returning NULL), before even reaching `malloc` (which would have failed and return NULL too).
      
      The implication is that we will not be able to support a single allocation of more than 2GB on a 32bit system (which i don't think is a realistic scenario).
      i.e. i do think we could be facing cases were redis consumes more than 2gb on a 32bit system, but not in a single allocation.
      
      The byproduct of this, is that i dropped the overflow assertions, since these will now lead to the same OOM panic we have for failed allocations.
      599e59eb
    • Binbin's avatar
      Add range check for server port in redis-cli/benchmark (#9854) · 20af95a9
      Binbin authored
      Validating inputs ahead of time, to give the end user a slightly more useful error.
      20af95a9
  13. 12 Jul, 2022 3 commits
  14. 11 Jul, 2022 5 commits
    • Binbin's avatar
      Trying to fix cluster test (#10963) · 693acc01
      Binbin authored
      #10942 break the new test added in #10449
      ```
      Testing unit: 29-slot-migration-response.tcl
      Cluster Join and auto-discovery test: FAILED: Cluster failed to join into a full mesh.
      ```
      
      It looks like we need to wait for the cluster in 28 to become stable.
      693acc01
    • Oran Agra's avatar
      update help.h (#10961) · b4568fc7
      Oran Agra authored
      b4568fc7
    • Binbin's avatar
      Add cluster-port support to redis-cli --cluster (#10344) · 35e8ae3e
      Binbin authored
      
      
      In #9389, we add a new `cluster-port` config and make cluster bus port configurable,
      and currently redis-cli --cluster create/add-node doesn't support with a configurable `cluster-port` instance.
      Because redis-cli uses the old way (port + 10000) to send the `CLUSTER MEET` command.
      
      Now we add this support on redis-cli `--cluster`, note we don't need to explicitly pass in the
      `cluster-port` parameter, we can get the real `cluster-port` of the node in `clusterManagerNodeLoadInfo`,
      so the `--cluster create` and `--cluster add-node` interfaces have not changed.
      
      We will use the `cluster-port` when we are doing `CLUSTER MEET`, also note that `CLUSTER MEET` bus-port
      parameter was added in 4.0, so if the bus_port (the one in redis-cli) is 0, or equal (port + 10000),
      we just call `CLUSTER MEET` with 2 arguments, using the old form.
      Co-authored-by: default avatarMadelyn Olson <34459052+madolson@users.noreply.github.com>
      35e8ae3e
    • Binbin's avatar
      Add --check-system in redis-server usage (#10960) · 1e85b89a
      Binbin authored
      1e85b89a
    • Madelyn Olson's avatar
      Fix crash during handshake and cluster shards call (#10942) · e6a1b2ea
      Madelyn Olson authored
      * Fix an engine crash when there are nodes in handshaking and a user calls cluster shards
      e6a1b2ea
  15. 10 Jul, 2022 1 commit
  16. 07 Jul, 2022 1 commit