1. 04 Oct, 2021 2 commits
    • Oran Agra's avatar
      Fix redis-cli / redis-sential overflow on some platforms (CVE-2021-32762) · 04ba4850
      Oran Agra authored
      The redis-cli command line tool and redis-sentinel service may be vulnerable
      to integer overflow when parsing specially crafted large multi-bulk network
      replies. This is a result of a vulnerability in the underlying hiredis
      library which does not perform an overflow check before calling the calloc()
      heap allocation function.
      
      This issue only impacts systems with heap allocators that do not perform their
      own overflow checks. Most modern systems do and are therefore not likely to
      be affected. Furthermore, by default redis-sentinel uses the jemalloc allocator
      which is also not vulnerable.
      04ba4850
    • Oran Agra's avatar
      Fix Integer overflow issue with intsets (CVE-2021-32687) · b1149a49
      Oran Agra authored
      The vulnerability involves changing the default set-max-intset-entries
      configuration parameter to a very large value and constructing specially
      crafted commands to manipulate sets
      b1149a49
  2. 21 Jul, 2021 35 commits
    • Oran Agra's avatar
      Redis 6.2.5 · db09f6eb
      Oran Agra authored
      db09f6eb
    • Huang Zhw's avatar
      On 32 bit platform, the bit position of GETBIT/SETBIT/BITFIELD/BITCOUNT,BITPOS... · 835d15b5
      Huang Zhw authored
      On 32 bit platform, the bit position of GETBIT/SETBIT/BITFIELD/BITCOUNT,BITPOS may overflow (see CVE-2021-32761) (#9191)
      
      GETBIT, SETBIT may access wrong address because of wrap.
      BITCOUNT and BITPOS may return wrapped results.
      BITFIELD may access the wrong address but also allocate insufficient memory and segfault (see CVE-2021-32761).
      
      This commit uses `uint64_t` or `long long` instead of `size_t`.
      related https://github.com/redis/redis/pull/8096
      
      At 32bit platform:
      > setbit bit 4294967295 1
      (integer) 0
      > config set proto-max-bulk-len 536870913
      OK
      > append bit "\xFF"
      (integer) 536870913
      > getbit bit 4294967296
      (integer) 0
      
      When the bit index is larger than 4294967295, size_t can't hold bit index. In the past,  `proto-max-bulk-len` is limit to 536870912, so there is no problem.
      
      After this commit, bit position is stored in `uint64_t` or `long long`. So when `proto-max-bulk-len > 536870912`, 32bit platforms can still be correct.
      
      For 64bit platform, this problem still exists. The major reason is bit pos 8 t...
      835d15b5
    • Oran Agra's avatar
      longer timeout in replication test (#8963) · bae0512c
      Oran Agra authored
      the test normally passes. but we saw one failure in a valgrind run in github actions
      
      (cherry picked from commit 8458baf6)
      bae0512c
    • Huang Zhw's avatar
      Remove testmodule in src/modules/Makefile. (#9250) · 8d613495
      Huang Zhw authored
      src/modules make failed. As in #3718 testmodule.c was removed. But the makefile was not updated
      
      (cherry picked from commit d54c9086)
      8d613495
    • Oran Agra's avatar
      Fix failing basics moduleapi test on 32bit CI (#9140) · 1d7c0e59
      Oran Agra authored
      
      (cherry picked from commit 5ffdbae1)
      1d7c0e59
    • Oran Agra's avatar
      Adjustments to recent RM_StringTruncate fix (#3718) (#9125) · 37b0f361
      Oran Agra authored
      - Introduce a new sdssubstr api as a building block for sdsrange.
        The API of sdsrange is many times hard to work with and also has
        corner case that cause bugs. sdsrange is easy to work with and also
        simplifies the implementation of sdsrange.
      - Revert the fix to RM_StringTruncate and just use sdssubstr instead of
        sdsrange.
      - Solve valgrind warnings from the new tests introduced by the previous
        PR.
      
      (cherry picked from commit ae418eca)
      37b0f361
    • Huang Zhw's avatar
      Fix missing separator in module info line (usedby and using lists) (#9241) · 68661171
      Huang Zhw authored
      
      
      Fix module info genModulesInfoStringRenderModulesList lack separator when there's more than one module in the list.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      (cherry picked from commit 1895e134)
      68661171
    • Binbin's avatar
      SMOVE only notify dstset when the addition is successful. (#9244) · b6225371
      Binbin authored
      in case dest key already contains the member, the dest key isn't modified, so the command shouldn't invalidate watch.
      
      (cherry picked from commit 11dc4e59)
      b6225371
    • qetu3790's avatar
      Set TCP keepalive on inbound clusterbus connections (#9230) · 355b1b6a
      qetu3790 authored
      Set TCP keepalive on inbound clusterbus connections to prevent memory leak
      
      (cherry picked from commit f03af47a)
      355b1b6a
    • Yossi Gottlieb's avatar
      Fix compatibility with OpenSSL 1.1.0. (#9233) · d6f42732
      Yossi Gottlieb authored
      
      (cherry picked from commit 277e4dc2)
      d6f42732
    • Oran Agra's avatar
      fix valgrind issues with recently added test in modules/blockonbackground (#9192) · de1b19ea
      Oran Agra authored
      fixes test issue introduced in #9167
      
      1. invalid reads due to accessing non-retained string (passed as unblock context).
      2. leaking module blocked client context, see #6922 for info.
      
      (cherry picked from commit a8518cce)
      de1b19ea
    • Yossi Gottlieb's avatar
      Fix CLIENT UNBLOCK crashing modules. (#9167) · 79fa5618
      Yossi Gottlieb authored
      Modules that use background threads with thread safe contexts are likely
      to use RM_BlockClient() without a timeout function, because they do not
      set up a timeout.
      
      Before this commit, `CLIENT UNBLOCK` would result with a crash as the
      `NULL` timeout callback is called. Beyond just crashing, this is also
      logically wrong as it may throw the module into an unexpected client
      state.
      
      This commits makes `CLIENT UNBLOCK` on such clients behave the same as
      any other client that is not in a blocked state and therefore cannot be
      unblocked.
      
      (cherry picked from commit aa139e2f)
      79fa5618
    • Huang Zhw's avatar
      redis-cli cluster import command may issue wrong MIGRATE command. (#8945) · 91bf2ab8
      Huang Zhw authored
      In clusterManagerCommandImport strcat was used to concat COPY and
      REPLACE, the space maybe not enough.
      If we use --cluster-replace but not --cluster-copy, the MIGRATE
      command contained COPY instead of REPLACE.
      
      (cherry picked from commit a049f629)
      91bf2ab8
    • Binbin's avatar
      Fix accidental deletion of sinterstore command when we meet wrong type error. (#9032) · 88655019
      Binbin authored
      SINTERSTORE would have deleted the dest key right away,
      even when later on it is bound to fail on an (WRONGTYPE) error.
      
      With this change it first picks up all the input keys, and only later
      delete the dest key if one is empty.
      
      Also add more tests for some commands.
      Mainly focus on
      - `wrong type error`:
      	expand test case (base on sinter bug) in non-store variant
      	add tests for store variant (although it exists in non-store variant, i think it would be better to have same tests)
      - the dstkey result when we meet `non-exist key (empty set)` in *store
      
      sdiff:
      - improve test case about wrong type error (the one we found in sinter, although it is safe in sdiff)
      - add test about using non-exist key (treat it like an empty set)
      sdiffstore:
      - according to sdiff test case, also add some tests about `wrong type error` and `non-exist key`
      - the different is that in sdiffstore, we will consider the `dstkey` result
      
      sunion/sunionstore add more tests (same as above)
      
      sinter/sinterstore also same as above ...
      
      (cherry picked from commit b8a5da80)
      88655019
    • Jason Elbaum's avatar
      Change return value type for ZPOPMAX/MIN in RESP3 (#8981) · fad44611
      Jason Elbaum authored
      When using RESP3, ZPOPMAX/ZPOPMIN should return nested arrays for consistency
      with other commands (e.g. ZRANGE).
      
      We do that only when COUNT argument is present (similarly to how LPOP behaves).
      for reasoning see https://github.com/redis/redis/issues/8824#issuecomment-855427955
      
      This is a breaking change only when RESP3 is used, and COUNT argument is present!
      
      (cherry picked from commit 7f342020)
      fad44611
    • Mikhail Fesenko's avatar
      Direct redis-cli repl prints to stderr, because --rdb can print to stdout.... · 88849712
      Mikhail Fesenko authored
      
      Direct redis-cli repl prints to stderr, because --rdb can print to stdout. fflush stdout after responses  (#9136)
      
      1. redis-cli can output --rdb data to stdout
         but redis-cli also write some messages to stdout which will mess up the rdb.
      
      2. Make redis-cli flush stdout when printing a reply
        This was needed in order to fix a hung in redis-cli test that uses
        --replica.
         Note that printf does flush when there's a newline, but fwrite does not.
      
      3. fix the redis-cli --replica test which used to pass previously
         because it didn't really care what it read, and because redis-cli
         used printf to print these other things to stdout.
      
      4. improve redis-cli --replica test to run with both diskless and disk-based.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      Co-authored-by: default avatarViktor Söderqvist <viktor@zuiderkwast.se>
      (cherry picked from commit 1eb4baa5)
      88849712
    • Rob Snyder's avatar
      Fix ziplist length updates on bigendian platforms (#2080) · 07e12486
      Rob Snyder authored
      Adds call to intrev16ifbe to ensure ZIPLIST_LENGTH is compared correctly
      
      (cherry picked from commit eaa52719)
      07e12486
    • Oran Agra's avatar
      Test infra, handle RESP3 attributes and big-numbers and bools (#9235) · 6cd84b64
      Oran Agra authored
      - promote the code in DEBUG PROTOCOL to addReplyBigNum
      - DEBUG PROTOCOL ATTRIB skips the attribute when client is RESP2
      - networking.c addReply for push and attributes generate assertion when
        called on a RESP2 client, anything else would produce a broken
        protocol that clients can't handle.
      
      (cherry picked from commit 6a5bac30)
      6cd84b64
    • Binbin's avatar
      hrandfield and zrandmember with count should return emptyarray when key does not exist. (#9178) · c6b3966d
      Binbin authored
      
      
      due to a copy-paste bug, it used to reply with null response rather than empty array.
      this commit includes new tests that are looking at the RESP response directly in
      order to be able to tell the difference between them.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      (cherry picked from commit a418a2d3)
      c6b3966d
    • Oran Agra's avatar
      Tests: add a way to read raw RESP protocol reponses (#9193) · 432e0566
      Oran Agra authored
      This makes it possible to distinguish between null response and an empty
      array (currently the tests infra translates both to an empty string/list)
      
      (cherry picked from commit 7103367a)
      432e0566
    • Mikhail Fesenko's avatar
      redis-cli --rdb: fix broken fsync/ftruncate for stdout (#9135) · bbb6c2fb
      Mikhail Fesenko authored
      
      
      A change in redis 6.2 caused redis-cli --rdb that's directed to stdout to fail because fsync fails.
      This commit avoids doing ftruncate (fails with a warning) and fsync (fails with an error) when the
      output file is `-`, and adds the missing documentation that `-` means stdout.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      Co-authored-by: default avatarWang Yuan <wangyuancode@163.com>
      (cherry picked from commit 74fe15b3)
      bbb6c2fb
    • Leibale Eidelman's avatar
      fix ZRANGESTORE - should return 0 when src points to an empty key (#9089) · 095a6e59
      Leibale Eidelman authored
      
      
      mistakenly it used to return an empty array rather than 0.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      (cherry picked from commit 95274f1f)
      095a6e59
    • guybe7's avatar
      Include sizeof(struct stream) in objectComputeSize (#9164) · 56020fd8
      guybe7 authored
      Affects MEMORY USAGE
      
      (cherry picked from commit 4434cebb)
      56020fd8
    • Binbin's avatar
      ZRANDMEMBER WITHSCORES with negative COUNT may return bad score (#9162) · 11235c0e
      Binbin authored
      Return a bad score when used with negative count (or count of 1), and non-ziplist encoded zset.
      Also add test to validate the return value and cover the issue.
      
      (cherry picked from commit 4bc5a832)
      11235c0e
    • Evan's avatar
      modules: Add newlen == 0 handling to RM_StringTruncate (#3717) (#3718) · 406caa5f
      Evan authored
      Previously, passing 0 for newlen would not truncate the string at all.
      This adds handling of this case, freeing the old string and creating a new empty string.
      
      Other changes:
      - Move `src/modules/testmodule.c` to `tests/modules/basics.c`
      - Introduce that basic test into the test suite
      - Add tests to cover StringTruncate
      - Add `test-modules` build target for the main makefile
      - Extend `distclean` build target to clean modules too
      
      (cherry picked from commit 1ccf2ca2)
      406caa5f
    • yvette903's avatar
      Fix coredump after Client Unpause command when threaded I/O is enabled (#9041) · 8bd5e391
      yvette903 authored
      Fix crash when using io-threads-do-reads and issuing CLIENT PAUSE and
      CLIENT UNPAUSE.
      This issue was introduced in redis 6.2 together with the FAILOVER command.
      
      (cherry picked from commit 096c5fd5)
      8bd5e391
    • guybe7's avatar
      Module API for current command name (#8792) · c547dff6
      guybe7 authored
      sometimes you can be very deep in the call stack, without access to argv.
      once you're there you may want your reply/log to contain the command name.
      
      (cherry picked from commit e16d3eb9)
      c547dff6
    • Huang Zhw's avatar
      Fix XTRIM or XADD with LIMIT may delete more entries than Count. (#9048) · e6df2f62
      Huang Zhw authored
      The decision to stop trimming due to LIMIT in XADD and XTRIM was after the limit was reached.
      i.e. the code was deleting **at least** that count of records (from the LIMIT argument's perspective, not the MAXLEN),
      instead of **up to** that count of records.
      see #9046
      
      (cherry picked from commit eaa7a7bb)
      e6df2f62
    • Huang Zhw's avatar
      XTRIM call streamParseAddOrTrimArgsOrReply use wrong arg xadd. (#9047) · 312ccae0
      Huang Zhw authored
      xtrimCommand call streamParseAddOrTrimArgsOrReply should use xadd==0.
      
      When the syntax is valid, it does not cause any bugs because the params of XADD is superset of XTRIM.
      Just XTRIM will not respond with error on invalid syntax. The syntax of XADD will also be accpeted by XTRIM.
      
      (cherry picked from commit 91f3689b)
      312ccae0
    • YaacovHazan's avatar
      stabilize tests that involved with load handlers (#8967) · ff272176
      YaacovHazan authored
      When test stop 'load handler' by killing the process that generating the load,
      some commands that already in the input buffer, still might be processed by the server.
      This may cause some instability in tests, that count on that no more commands
      processed after we stop the `load handler'
      
      In this commit, new proc 'wait_load_handlers_disconnected' added, to verify that no more
      cammands from any 'load handler' prossesed, by checking that the clients who
      genreate the load is disconnceted.
      
      Also, replacing check of dbsize with wait_for_ofs_sync before comparing debug digest, as
      it would fail in case the last key the workload wrote was an overridden key (not a new one).
      
      Affected tests
      Race fix:
      - failover command to specific replica works
      - Connect multiple replicas at the same time (issue #141), master diskless=$mdl, replica diskless=$sdl
      - AOF rewrite during write load: RDB preamble=$rdbpre
      
      Cleanup and speedup:
      - Test replication with blocking lists and sorted sets operations
      - Test replication with parallel clients writing in different DBs
      - Test replication partial resync: $descr (diskless: $mdl, $sdl, reconnect: $reconnect
      
      (cherry picked from commit 32a2584e)
      ff272176
    • perryitay's avatar
      Fail EXEC command in case a watched key is expired (#9194) · 3f4f9b63
      perryitay authored
      
      
      There are two issues fixed in this commit: 
      1. we want to fail the EXEC command in case there is a watched key that's logically
         expired but not yet deleted by active expire or lazy expire.
      2. we saw that currently cache time is update in every `call()` (including nested calls),
         this time is being also being use for the isKeyExpired comparison, we want to update
         the cache time only in the first call (execCommand)
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      (cherry picked from commit ac8b1df8)
      3f4f9b63
    • Huang Zhw's avatar
      Do not install a file event to send data to rewrite child when parent stop... · 5b8e3951
      Huang Zhw authored
      Do not install a file event to send data to rewrite child when parent stop sending diff to child in aof rewrite. (#8767)
      
      In aof rewrite, when parent stop sending data to child, if there is
      new rewrite data, aofChildWriteDiffData write event will be installed.
      Then this event is issued and deletes the file event without do anyting.
      This will happen over and over again until aof rewrite finish.
      
      This bug used to waste a few system calls per excessive wake-up
      (epoll_ctl and epoll_wait) per cycle, each cycle triggered by receiving
      a write command from a client.
      
      (cherry picked from commit cb961d8c)
      5b8e3951
    • Omer Shadmi's avatar
      Avoid exiting to allow diskless loading to recover from RDB short read on module AUX data (#9199) · 9914b676
      Omer Shadmi authored
      Currently a replica is able to recover from a short read (when diskless loading
      is enabled) and avoid crashing/exiting, replying to the master and then the rdb
      could be sent again by the master for another load attempt by the replica.
      There were a few scenarios that were not behaving similarly, such as when
      there is no end-of-file marker, or when module aux data failed to load, which
      should be allowed to occur due to a short read.
      
      (cherry picked from commit f06d782f)
      9914b676
    • Oran Agra's avatar
      Fix race in client side tracking (#9116) · abd44c83
      Oran Agra authored
      The `Tracking gets notification of expired keys` test in tracking.tcl
      used to hung in valgrind CI quite a lot.
      
      It turns out the reason is that with valgrind and a busy machine, the
      server cron active expire cycle could easily run in the same event loop
      as the command that created `mykey`, so that when they key got expired,
      there were two change events to broadcast, one that set the key and one
      that expired it, but since we used raxTryInsert, the client that was
      associated with the "last" change was the one that created the key, so
      the NOLOOP filtered that event.
      
      This commit adds a test that reproduces the problem by using lazy expire
      in a multi-exec which makes sure the key expires in the same event loop
      as the one that added it.
      
      (cherry picked from commit 9b564b52)
      abd44c83
    • Maxim Galushka's avatar
      redis-cli: support for REDIS_REPLY_SET in CSV and RAW output. (#7338) · f70916f8
      Maxim Galushka authored
      Fixes #6792. Added support of REDIS_REPLY_SET in raw and csv output of `./redis-cli`
      
      Test:
      
      run commands to test:
        ./redis-cli -3 --csv COMMAND
        ./redis-cli -3 --raw COMMAND
      
      Now they are returning resuts, were failing with: "Unknown reply type: 10" before the change.
      
      (cherry picked from commit 96bb0785)
      f70916f8
  3. 01 Jun, 2021 3 commits