1. 12 Dec, 2022 17 commits
    • Oran Agra's avatar
      fixes for fork child exit and test: #11463 (#11499) · 4ac3d79b
      Oran Agra authored
      
      
      Fix a few issues with the recent #11463
      * use exitFromChild instead of exit
      * test should ignore defunct process since that's what we expect to
        happen for thees child processes when the parent dies.
      * fix typo
      Co-authored-by: default avatarBinbin <binloveplay1314@qq.com>
      (cherry picked from commit 4c54528f)
      4ac3d79b
    • Oran Agra's avatar
      diskless master, avoid bgsave child hung when fork parent crashes (#11463) · 51fa40ff
      Oran Agra authored
      During a diskless sync, if the master main process crashes, the child would
      have hung in `write`. This fix closes the read fd on the child side, so that if the
      parent crashes, the child will get a write error and exit.
      
      This change also fixes disk-based replication, BGSAVE and AOFRW.
      In that case the child wouldn't have been hang, it would have just kept
      running until done which may be pointless.
      
      There is a certain degree of risk here. in case there's a BGSAVE child that could
      maybe succeed and the parent dies for some reason, the old code would have let
      the child keep running and maybe succeed and avoid data loss.
      On the other hand, if the parent is restarted, it would have loaded an old rdb file
      (or none), and then the child could reach the end and rename the rdb file (data
      conflicting with what the parent has), or also have a race with another BGSAVE
      child that the new parent started.
      
      Note that i removed a comment saying a write error will be ignored in the child
      and handled by the parent (this comment was very old and i don't think relevant).
      
      (cherry picked from commit ccaef5c9)
      51fa40ff
    • Moti Cohen's avatar
      Fix sentinel function that compares hostnames (if failed resolve) (#11419) · 042512aa
      Moti Cohen authored
      Funcion sentinelAddrEqualsHostname() of sentinel makes DNS resolve
      and based on it determines if two IP addresses are equal. Now, If the
      DNS resolve command fails, the function simply returns 0, even if the
      hostnames are identical.
      
      This might become an issue in case of failover such that sentinel might
      receives from Redis instance, response to regular INFO query it sent,
      and wrongly decide that the instance is pointing to is different leader
      than the one recorded because of this function, yet hostnames are
      identical. In turn sentinel disconnects the connection between sentinel
      and valid slave which leads to -failover-abort-no-good-slave.
      See issue #11241.
      
      I managed to reproduce only part of the flow in which the function
      return wrong result and trigger +fix-slave-config.
      
      The fix is even if the function failed to resolve then compare based on
      hostnames. That is our best effort as long as the server is unavailable
      for some reason. It is fine since Redis instance cannot have multiple
      hostnames for a given setup
      
      (cherry picked from commit bd23b15a)
      042512aa
    • Oran Agra's avatar
      Improve linux overcommit check and warning (#11357) · 3f39c18a
      Oran Agra authored
      1. show the overcommit warning when overcommit is disabled (2),
         not just when it is set to heuristic (0).
      2. improve warning text to mention the issue with jemalloc causing VM
         mapping fragmentation when set to 2.
      
      (cherry picked from commit dd60c6c8)
      3f39c18a
    • Meir Shpilraien (Spielrein)'s avatar
      Avoid crash on crash report when a bad function pointer was called (#11298) · d8102f5f
      Meir Shpilraien (Spielrein) authored
      If Redis crashes due to calling an invalid function pointer,
      the `backtrace` function will try to dereference this invalid pointer
      which will cause a crash inside the crash report and will kill
      the processes without having all the crash report information.
      
      Example:
      
      ```
      === REDIS BUG REPORT START: Cut & paste starting from here ===
      198672:M 19 Sep 2022 18:06:12.936 # Redis 255.255.255 crashed by signal: 11, si_code: 1
      198672:M 19 Sep 2022 18:06:12.936 # Accessing address: 0x1
      198672:M 19 Sep 2022 18:06:12.936 # Crashed running the instruction at: 0x1
      // here the processes is crashing
      ```
      
      This PR tries to fix this crash be:
      1. Identify the issue when it happened.
      2. Replace the invalid pointer with a pointer to some dummy function
         so that `backtrace` will not crash.
      
      I identification is done by comparing `eip` to `info->si_addr`, if they
      are the same we know that the crash happened on the same address it tries to
      accesses and we can conclude that it tries to call and invalid function pointer.
      
      To replace the invalid pointer we introduce a new function, `setMcontextEip`,
      which is very similar to `getMcontextEip` and it knows to set the Eip for the
      different supported OS's. After printing the trace we retrieve the old `Eip` value.
      
      (cherry picked from commit 0bf90d94)
      d8102f5f
    • zhaozhao.zz's avatar
      fix infinite sleep in performEvictions when have lazyfree jobs (#11237) · 65ad4ff8
      zhaozhao.zz authored
      This bug is introduced in #7653. (Redis 6.2.0)
      
      When `server.maxmemory_eviction_tenacity` is 100, `eviction_time_limit_us` is
      `ULONG_MAX`, and if we cannot find the best key to delete (e.g. maxmemory-policy
      is `volatile-lru` and all keys with ttl have been evicted), in `cant_free` redis will sleep
      forever if some items are being freed in the lazyfree thread.
      
      (cherry picked from commit 464aa041)
      65ad4ff8
    • DarrenJiang13's avatar
      fix the client type in trackingInvalidateKey() (#11052) · ec5564a4
      DarrenJiang13 authored
      Fix bug with scripts ignoring client tracking NOLOOP and
      send an invalidation message anyway.
      
      (cherry picked from commit 44859a41)
      ec5564a4
    • Huang Zhw's avatar
      tracking pending invalidation message of flushdb sent by (#11068) · 2cd5f6f3
      Huang Zhw authored
      trackingHandlePendingKeyInvalidations should use proto.
      
      (cherry picked from commit 61451b02)
      2cd5f6f3
    • Huang Zhw's avatar
      When client tracking is on, invalidation message of flushdb in a (#11038) · fbcd591b
      Huang Zhw authored
      When FLUSHDB / FLUSHALL / SWAPDB is inside MULTI / EXEC, the
      client side tracking invalidation message was interleaved with transaction response.
      
      (cherry picked from commit 6f0a27e3)
      fbcd591b
    • Meir Shpilraien (Spielrein)'s avatar
      Fix #11030, use lua_rawget to avoid triggering metatables and crash. (#11032) · e1b17f12
      Meir Shpilraien (Spielrein) authored
      Fix #11030, use lua_rawget to avoid triggering metatables.
      
      #11030 shows how return `_G` from the Lua script (either function or eval), cause the
      Lua interpreter to Panic and the Redis processes to exit with error code 1.
      Though return `_G` only panic on Redis 7 and 6.2.7, the underline issue exists on older
      versions as well (6.0 and 6.2). The underline issue is returning a table with a metatable
      such that the metatable raises an error.
      
      The following example demonstrate the issue:
      ```
      127.0.0.1:6379> eval "local a = {}; setmetatable(a,{__index=function() foo() end}) return a" 0
      Error: Server closed the connection
      ```
      ```
      PANIC: unprotected error in call to Lua API (user_script:1: Script attempted to access nonexistent global variable 'foo')
      ```
      
      The Lua panic happened because when returning the result to the client, Redis needs to
      introspect the returning table and transform the table into a resp. In order to scan the table,
      Redis uses `lua_gettable` api which might trigger the metatable (if exists) and might raise an error.
      This code is not running inside `pcall` (Lua protected call), so raising an error causes the
      Lua to panic and exit. Notice that this is not a crash, its a Lua panic that exit with error code 1.
      
      Returning `_G` panics on Redis 7 and 6.2.7 because on those versions `_G` has a metatable
      that raises error when trying to fetch a none existing key.
      
      ### Solution
      
      Instead of using `lua_gettable` that might raise error and cause the issue, use `lua_rawget`
      that simply return the value from the table without triggering any metatable logic.
      This is promised not to raise and error.
      
      The downside of this solution is that it might be considered as breaking change, if someone
      rely on metatable in the returned value. An alternative solution is to wrap this entire logic
      with `pcall` (Lua protected call), this alternative require a much bigger refactoring.
      
      ### Back Porting
      
      The same fix will work on older versions as well (6.2, 6.0). Notice that on those version,
      the issue can cause Redis to crash if inside the metatable logic there is an attempt to accesses
      Redis (`redis.call`). On 7.0, there is not crash and the `redis.call` is executed as if it was done
      from inside the script itself.
      
      ### Tests
      
      Tests was added the verify the fix
      
      (cherry picked from commit 020e046b)
      e1b17f12
    • chenyang8094's avatar
      Register abs-expire apis (#11025) · 8c76e0f2
      chenyang8094 authored
      RM_SetAbsExpire and RM_GetAbsExpire were not actually operational since
      they were introduced, due to omission in API registration.
      
      (cherry picked from commit 39d216a3)
      8c76e0f2
    • Yossi Gottlieb's avatar
      TLS: Notify clients on connection shutdown. (#10931) · b4de9e83
      Yossi Gottlieb authored
      Use SSL_shutdown(), in a best-effort manner, when closing a TLS
      connection. This change better supports OpenSSL 3.x clients that will
      not silently ignore the socket-level EOF.
      
      (cherry picked from commit 45ae6053)
      b4de9e83
    • Oran Agra's avatar
      optimize zset conversion on large ZRANGESTORE (#10789) · 70aaf500
      Oran Agra authored
      when we know the size of the zset we're gonna store in advance,
      we can check if it's greater than the listpack encoding threshold,
      in which case we can create a skiplist from the get go, and avoid
      converting the listpack to skiplist later after it was already populated.
      
      (cherry picked from commit 21891003)
      70aaf500
    • Oran Agra's avatar
      crash report instructions (#10816) · 66472a5e
      Oran Agra authored
      Trying to avoid people opening crash report issues about module crashes and ARM QEMU bugs.
      
      (cherry picked from commit 475563e2)
      66472a5e
    • Vitaly's avatar
      Fix ZRANGESTORE crash when zset_max_listpack_entries is 0 (#10767) · fd7bde57
      Vitaly authored
      When `zrangestore` is called container destination object is created.
      Before this PR we used to create a listpack based object even if `zset-max-ziplist-entries`
      or equivalent`zset-max-listpack-entries` was set to 0.
      This triggered immediate conversion of the listpack into a skiplist in `zrangestore`, which hits
      an assertion resulting in an engine crash.
      
      Added a TCL test that reproduces this issue.
      
      (cherry picked from commit 6461f09f)
      fd7bde57
    • Madelyn Olson's avatar
      Unpause clients earlier during manual cluster failover (#9676) · 9d40f1cb
      Madelyn Olson authored
      Unpause clients after manual failover ends instead of the timed offset
      
      (cherry picked from commit 32215e78)
      9d40f1cb
    • Wen Hui's avatar
      Sentinel: don't log auth-pass value for better security (#9652) · 389c5e14
      Wen Hui authored
      (cherry picked from commit 43b22f17)
      389c5e14
  2. 06 Dec, 2022 1 commit
    • Meir Shpilraien (Spielrein)'s avatar
      Remove dead code on sorting reply on Lua scripts. (#10701) · 64c657a8
      Meir Shpilraien (Spielrein) authored
      On v6.2.7 a new mechanism was added to Lua scripts that allows
      filtering the globals of the Lua interpreter. This mechanism was
      added on the following commit: https://github.com/redis/redis/commit/11b602fbf8f9cdf8fc741c24625ab6287ab998a9
      
      One of the globals that was filtered out was `__redis__compare_helper`. This global
      was missed and was not added to the allow list or to the deny list. This is
      why we get the following warning when Redis starts:
      `A key '__redis__compare_helper' was added to Lua globals which is not on the globals allow list nor listed on the deny list.`
      
      After investigating the git blame log, the conclusion is that `__redis__compare_helper`
      is no longer needed, the PR deletes this function, and fixes the warning.
      
      Detailed Explanation:
      
      `__redis__compare_helper` was added on this commit: https://github.com/redis/redis/commit/2c861050c1
      Its purpose is to sort the replies of `SORT` command when script replication is enable and keep the replies
      deterministic and avoid primary and replica synchronization issues. On `SORT` command, there was a need for
      special compare function that are able to compare boolean values.
      
      The need to sort the `SORT` command reply was removed on this commit: https://github.com/redis/redis/commit/36741b2c818a95e8ef167818271614ee6b1bc414
      The sorting was moved to be part of the `SORT` command and there was not longer a need
      to sort it on the Lua interpreter. The commit made `__redis__compare_helper` a dead code but did
      not deleted it.
      64c657a8
  3. 27 Apr, 2022 22 commits