1. 11 Nov, 2021 2 commits
    • Ozan Tezcan's avatar
      Add sanitizer support and clean up sanitizer findings (#9601) · b91d8b28
      Ozan Tezcan authored
      - Added sanitizer support. `address`, `undefined` and `thread` sanitizers are available.  
      - To build Redis with desired sanitizer : `make SANITIZER=undefined`
      - There were some sanitizer findings, cleaned up codebase
      - Added tests with address and undefined behavior sanitizers to daily CI.
      - Added tests with address sanitizer to the per-PR CI (smoke out mem leaks sooner).
      
      Basically, there are three types of issues : 
      
      **1- Unaligned load/store** : Most probably, this issue may cause a crash on a platform that
      does not support unaligned access. Redis does unaligned access only on supported platforms.
      
      **2- Signed integer overflow.** Although, signed overflow issue can be problematic time to time
      and change how compiler generates code, current findings mostly about signed shift or simple
      addition overflow. For most platforms Redis can be compiled for, this wouldn't cause any issue
      as far as I can tell (checked generated code on godbolt.org).
      
       **3 -Minor leak** (redis-cli), **use-after-free**(just before calling exit());
      
      UB means nothing guaranteed and risky to reason about program behavior but I don't think any
      of the fixes here worth backporting. As sanitizers are now part of the CI, preventing new issues
      will be the real benefit. 
      b91d8b28
    • yoav-steinberg's avatar
      Archive external redis log in external tests (#9765) · cd6b3d55
      yoav-steinberg authored
      On test failure store the external redis server logs as CI artifacts so we can review them.
      
      Write test name to server log for external server tests.
      This is attempted and silently failed in case external server doesn't support it.
      Note that in non-external server mode we use a more robust method of writing to the log which doesn't depend on the
      server actually running/working. This isn't possible for externl servers and required for some complex tests which are
      skipped in external mode anyway.
      
      Cleanup: remove dup code.
      cd6b3d55
  2. 07 Nov, 2021 1 commit
  3. 03 Nov, 2021 1 commit
    • perryitay's avatar
      Add support for list type to store elements larger than 4GB (#9357) · f27083a4
      perryitay authored
      
      
      Redis lists are stored in quicklist, which is currently a linked list of ziplists.
      Ziplists are limited to storing elements no larger than 4GB, so when bigger
      items are added they're getting truncated.
      This PR changes quicklists so that they're capable of storing large items
      in quicklist nodes that are plain string buffers rather than ziplist.
      
      As part of the PR there were few other changes in redis: 
      1. new DEBUG sub-commands: 
         - QUICKLIST-PACKED-THRESHOLD - set the threshold of for the node type to
           be plan or ziplist. default (1GB)
         - QUICKLIST <key> - Shows low level info about the quicklist encoding of <key>
      2. rdb format change:
         - A new type was added - RDB_TYPE_LIST_QUICKLIST_2 . 
         - container type (packed / plain) was added to the beginning of the rdb object
           (before the actual node list).
      3. testing:
         - Tests that requires over 100MB will be by default skipped. a new flag was
           added to 'runtest' to run the large memory tests (not used by default)
      Co-authored-by: default avatarsundb <sundbcn@gmail.com>
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      f27083a4
  4. 02 Nov, 2021 1 commit
    • Oran Agra's avatar
      attempt to fix tracking test issue with external tests due to lazy free (#9722) · 87321deb
      Oran Agra authored
      The External tests started failing recently for unclear reason:
      ```
      *** [err]: Tracking invalidation message of eviction keys should be before response in tests/unit/tracking.tcl
      Expected '0' to be equal to 'invalidate volatile-key' (context: type eval line 21 cmd {assert_equal $res {invalidate volatile-key}} proc ::test)
      ```
      
      I suspect the issue is that the used_memory sample is taken while a lazy free is still being processed.
      87321deb
  5. 26 Oct, 2021 1 commit
    • Oran Agra's avatar
      Testsuite: attempt to find / avoid valgrind warnings of killed processes (#9679) · 665e4284
      Oran Agra authored
      I recently started seeing a lot of empty valgrind reports in the daily CI.
      i.e. prints showing valgrind header but no leak report, which causes the tests to fail
      https://github.com/redis/redis/runs/3991335416?check_suite_focus=true
      
      This commit change 2 things:
      * first, considering valgrind is just slow, we used to give processes 60 seconds timeout on shutdown
        instead of 10 seconds we give normally. this commit changes that to 120.
      * secondly, when we reach the timeout, we first try to use SIGSEGV so that maybe we'll get a stack
        trace indicating where redis is hang, and we only resort to SIGKILL if double that time passed.
      
      note that if there are indeed hang processes, we will normally not see that in the non-valgrind runs,
      since the tests didn't use to detect any failure in that case, and now they will since `crashlog_from_file`
      is run after `kill_server`.
      665e4284
  6. 26 Sep, 2021 2 commits
    • yoav-steinberg's avatar
      Client eviction ci issues (#9549) · 66002530
      yoav-steinberg authored
      Fixing CI test issues introduced in #8687
      - valgrind warnings in readQueryFromClient when client was freed by processInputBuffer
      - adding DEBUG pause-cron for tests not to be time dependent.
      - skipping a test that depends on socket buffers / events not compatible with TLS
      - making sure client got subscribed by not using deferring client
      66002530
    • Yossi Gottlieb's avatar
      Add --skipfile and --skiptest regex support. (#9555) · 0af7fe2c
      Yossi Gottlieb authored
      Empty patterns are not considered and skipped.
      Also, improve help text.
      0af7fe2c
  7. 24 Sep, 2021 1 commit
  8. 23 Sep, 2021 1 commit
    • yoav-steinberg's avatar
      Client eviction (#8687) · 2753429c
      yoav-steinberg authored
      
      
      ### Description
      A mechanism for disconnecting clients when the sum of all connected clients is above a
      configured limit. This prevents eviction or OOM caused by accumulated used memory
      between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
      mechanism which takes into account not only a single client and not only output buffers
      but rather all memory used by all clients.
      
      #### Design
      The general design is as following:
      * We track memory usage of each client, taking into account all memory used by the
        client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
        after reading from the socket, after processing commands and after writing to the socket.
      * Based on the used memory we sort all clients into buckets. Each bucket contains all
        clients using up up to x2 memory of the clients in the bucket below it. For example up
        to 1m clients, up to 2m clients, up to 4m clients, ...
      * Before processing a command and before sleep we check if we're over the configured
        limit. If we are we start disconnecting clients from larger buckets downwards until we're
        under the limit.
      
      #### Config
      `maxmemory-clients` max memory all clients are allowed to consume, above this threshold
      we disconnect clients.
      This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
      suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
      would mean 10% of `maxmemory`).
      
      #### Important code changes
      * During the development I encountered yet more situations where our io-threads access
        global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
        memory buckets (which are global) while their memory usage changes in the io-thread.
        To achieve this I decided to simplify how we check if we're in an io-thread and make it
        much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
        if the client is in an io-thread (it wasn't used for anything else) and just used the global
        `io_threads_op` variable the same way to check during writes.
      * I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
        We now store a pointer in the `client` struct to this list so we don't need to search in it
        (`pending_read_list_node`).
      * Added `evicted_clients` stat to `INFO` command.
      * Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
        client eviction mechanism. Added corrosponding 'e' flag in the client info string.
      * Added `multi-mem` field in the client info string to show how much memory is used up
        by buffered multi commands.
      * Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
        channels (partially), tracking prefixes (partially).
      * CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
        clients will be disconnected between processing different clients and not only before sleep.
        This new function can be used in the future for work we want to do outside the command
        processing loop but don't want to wait for all clients to be processed before we get to it.
        Specifically I wanted to handle output-buffer-limit related closing before we process client
        eviction in case the two race with each other.
      * Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
        buckets.
      * Each client now holds a pointer to the client eviction memory usage bucket it belongs to
        and listNode to itself in that bucket for quick removal.
      * Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
        indicating no io-threading is currently being executed.
      * In order to track memory used by each clients in real-time we can't rely on updating
        these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
        (used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
        writing data to pubsub clients, after writing the output buffer and after reading from the
        socket (and maybe other places too). The function is written to be fast.
      * Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
        processing a command (before performing oom-checks and key-eviction).
      * All clients memory usage buckets are grouped as follows:
        * All clients using less than 64k.
        * 64K..128K
        * 128K..256K
        * ...
        * 2G..4G
        * All clients using 4g and up.
      * Added client-eviction.tcl with a bunch of tests for the new mechanism.
      * Extended maxmemory.tcl to test the interaction between maxmemory and
        maxmemory-clients settings.
      * Added an option to flag a numeric configuration variable as a "percent", this means that
        if we encounter a '%' after the number in the config file (or config set command) we
        consider it as valid. Such a number is store internally as a negative value. This way an
        integer value can be interpreted as either a percent (negative) or absolute value (positive).
        This is useful for example if some numeric configuration can optionally be set to a percentage
        of something else.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      2753429c
  9. 14 Sep, 2021 1 commit
  10. 13 Sep, 2021 1 commit
  11. 12 Sep, 2021 1 commit
    • Huang Zhw's avatar
      bitpos/bitcount add bit index (#9324) · 75dd2309
      Huang Zhw authored
      Make bitpos/bitcount support bit index:
      
      ```
      BITPOS key bit [start [end [BIT|BYTE]]]
      BITCOUNT key [start end [BIT|BYTE]]
      ```
      
      The default behavior is `BYTE`, so these commands are still compatible with old.
      75dd2309
  12. 10 Aug, 2021 1 commit
  13. 05 Aug, 2021 1 commit
    • Oran Agra's avatar
      corrupt-dump-fuzzer test, avoid creating junk keys (#9302) · 3f3f678a
      Oran Agra authored
      The execution of the RPOPLPUSH command by the fuzzer created junk keys,
      that were later being selected by RANDOMKEY and modified.
      This also meant that lists were statistically tested more than other
      files.
      
      Fix the fuzzer not to pass junk key names to RPOPLPUSH, and add a check
      that detects that new keys are not added by the fuzzer to detect future
      similar issues.
      3f3f678a
  14. 04 Aug, 2021 1 commit
    • Meir Shpilraien (Spielrein)'s avatar
      Unified Lua and modules reply parsing and added RESP3 support to RM_Call (#9202) · 2237131e
      Meir Shpilraien (Spielrein) authored
      
      
      ## Current state
      1. Lua has its own parser that handles parsing `reds.call` replies and translates them
        to Lua objects that can be used by the user Lua code. The parser partially handles
        resp3 (missing big number, verbatim, attribute, ...)
      2. Modules have their own parser that handles parsing `RM_Call` replies and translates
        them to RedisModuleCallReply objects. The parser does not support resp3.
      
      In addition, in the future, we want to add Redis Function (#8693) that will probably
      support more languages. At some point maintaining so many parsers will stop
      scaling (bug fixes and protocol changes will need to be applied on all of them).
      We will probably end up with different parsers that support different parts of the
      resp protocol (like we already have today with Lua and modules)
      
      ## PR Changes
      This PR attempt to unified the reply parsing of Lua and modules (and in the future
      Redis Function) by introducing a new parser unit (`resp_parser.c`). The new parser
      handles parsing the reply and calls different callbacks to allow the users (another
      unit that uses the parser, i.e, Lua, modules, or Redis Function) to analyze the reply.
      
      ### Lua API Additions
      The code that handles reply parsing on `scripting.c` was removed. Instead, it uses
      the resp_parser to parse and create a Lua object out of the reply. As mentioned
      above the Lua parser did not handle parsing big numbers, verbatim, and attribute.
      The new parser can handle those and so Lua also gets it for free.
      Those are translated to Lua objects in the following way:
      1. Big Number - Lua table `{'big_number':'<str representation for big number>'}`
      2. Verbatim - Lua table `{'verbatim_string':{'format':'<verbatim format>', 'string':'<verbatim string value>'}}`
      3. Attribute - currently ignored and not expose to the Lua parser, another issue will be open to decide how to expose it.
      
      Tests were added to check resp3 reply parsing on Lua
      
      ### Modules API Additions
      The reply parsing code on `module.c` was also removed and the new resp_parser is used instead.
      In addition, the RedisModuleCallReply was also extracted to a separate unit located on `call_reply.c`
      (in the future, this unit will also be used by Redis Function). A nice side effect of unified parsing is
      that modules now also support resp3. Resp3 can be enabled by giving `3` as a parameter to the
      fmt argument of `RM_Call`. It is also possible to give `0`, which will indicate an auto mode. i.e, Redis
      will automatically chose the reply protocol base on the current client set on the RedisModuleCtx
      (this mode will mostly be used when the module want to pass the reply to the client as is).
      In addition, the following RedisModuleAPI were added to allow analyzing resp3 replies:
      
      * New RedisModuleCallReply types:
         * `REDISMODULE_REPLY_MAP`
         * `REDISMODULE_REPLY_SET`
         * `REDISMODULE_REPLY_BOOL`
         * `REDISMODULE_REPLY_DOUBLE`
         * `REDISMODULE_REPLY_BIG_NUMBER`
         * `REDISMODULE_REPLY_VERBATIM_STRING`
         * `REDISMODULE_REPLY_ATTRIBUTE`
      
      * New RedisModuleAPI:
         * `RedisModule_CallReplyDouble` - getting double value from resp3 double reply
         * `RedisModule_CallReplyBool` - getting boolean value from resp3 boolean reply
         * `RedisModule_CallReplyBigNumber` - getting big number value from resp3 big number reply
         * `RedisModule_CallReplyVerbatim` - getting format and value from resp3 verbatim reply
         * `RedisModule_CallReplySetElement` - getting element from resp3 set reply
         * `RedisModule_CallReplyMapElement` - getting key and value from resp3 map reply
         * `RedisModule_CallReplyAttribute` - getting a reply attribute
         * `RedisModule_CallReplyAttributeElement` - getting key and value from resp3 attribute reply
         
      * New context flags:
         * `REDISMODULE_CTX_FLAGS_RESP3` - indicate that the client is using resp3
      
      Tests were added to check the new RedisModuleAPI
      
      ### Modules API Changes
      * RM_ReplyWithCallReply might return REDISMODULE_ERR if the given CallReply is in resp3
        but the client expects resp2. This is not a breaking change because in order to get a resp3
        CallReply one needs to specifically specify `3` as a parameter to the fmt argument of
        `RM_Call` (as mentioned above).
      
      Tests were added to check this change
      
      ### More small Additions
      * Added `debug set-disable-deny-scripts` that allows to turn on and off the commands no-script
      flag protection. This is used by the Lua resp3 tests so it will be possible to run `debug protocol`
      and check the resp3 parsing code.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      Co-authored-by: default avatarYossi Gottlieb <yossigo@gmail.com>
      2237131e
  15. 03 Aug, 2021 1 commit
  16. 29 Jul, 2021 1 commit
    • Wen Hui's avatar
      Remove duplicate zero-port sentinels (#9240) · db415364
      Wen Hui authored
      The issue is that when a sentinel with the same address and IP is turned on with a different runid, its port is set to 0 but it is still present in the dictionary master->sentinels which contain all the sentinels for a master.
      
      This causes a problem when we do INFO SENTINEL because it takes the size of the dictionary of sentinels. This might also cause a problem for failover if enough sentinels have their port set to 0 since the number of voters in failover is also determined by the size of the dictionary of sentinels.
      
      This commits removes the sentinels with the port set to zero from the dictionary of sentinels.
      Fixes #8786
      db415364
  17. 14 Jul, 2021 1 commit
    • Oran Agra's avatar
      Test infra, handle RESP3 attributes and big-numbers and bools (#9235) · 6a5bac30
      Oran Agra authored
      - promote the code in DEBUG PROTOCOL to addReplyBigNum
      - DEBUG PROTOCOL ATTRIB skips the attribute when client is RESP2
      - networking.c addReply for push and attributes generate assertion when
        called on a RESP2 client, anything else would produce a broken
        protocol that clients can't handle.
      6a5bac30
  18. 04 Jul, 2021 1 commit
  19. 22 Jun, 2021 1 commit
    • Yossi Gottlieb's avatar
      Improve bind and protected-mode config handling. (#9034) · 07b0d144
      Yossi Gottlieb authored
      * Specifying an empty `bind ""` configuration prevents Redis from listening on any TCP port. Before this commit, such configuration was not accepted.
      * Using `CONFIG GET bind` will always return an explicit configuration value. Before this commit, if a bind address was not specified the returned value was empty (which was an anomaly).
      
      Another behavior change is that modifying the `bind` configuration to a non-default value will NO LONGER DISABLE protected-mode implicitly.
      07b0d144
  20. 10 Jun, 2021 1 commit
    • Binbin's avatar
      Fixed some typos, add a spell check ci and others minor fix (#8890) · 0bfccc55
      Binbin authored
      This PR adds a spell checker CI action that will fail future PRs if they introduce typos and spelling mistakes.
      This spell checker is based on blacklist of common spelling mistakes, so it will not catch everything,
      but at least it is also unlikely to cause false positives.
      
      Besides that, the PR also fixes many spelling mistakes and types, not all are a result of the spell checker we use.
      
      Here's a summary of other changes:
      1. Scanned the entire source code and fixes all sorts of typos and spelling mistakes (including missing or extra spaces).
      2. Outdated function / variable / argument names in comments
      3. Fix outdated keyspace masks error log when we check `config.notify-keyspace-events` in loadServerConfigFromString.
      4. Trim the white space at the end of line in `module.c`. Check: https://github.com/redis/redis/pull/7751
      5. Some outdated https link URLs.
      6. Fix some outdated comment. Such as:
          - In README: about the rdb, we used to said create a `thread`, change to `process`
          - dbRandomKey function coment (about the dictGetRandomKey, change to dictGetFairRandomKey)
          - notifyKeyspaceEvent fucntion comment (add type arg)
          - Some others minor fix in comment (Most of them are incorrectly quoted by variable names)
      7. Modified the error log so that users can easily distinguish between TCP and TLS in `changeBindAddr`
      0bfccc55
  21. 09 Jun, 2021 1 commit
    • Yossi Gottlieb's avatar
      Improve test suite to handle external servers better. (#9033) · 8a86bca5
      Yossi Gottlieb authored
      This commit revives the improves the ability to run the test suite against
      external servers, instead of launching and managing `redis-server` processes as
      part of the test fixture.
      
      This capability existed in the past, using the `--host` and `--port` options.
      However, it was quite limited and mostly useful when running a specific tests.
      Attempting to run larger chunks of the test suite experienced many issues:
      
      * Many tests depend on being able to start and control `redis-server` themselves,
      and there's no clear distinction between external server compatible and other
      tests.
      * Cluster mode is not supported (resulting with `CROSSSLOT` errors).
      
      This PR cleans up many things and makes it possible to run the entire test suite
      against an external server. It also provides more fine grained controls to
      handle cases where the external server supports a subset of the Redis commands,
      limited number of databases, cluster mode, etc.
      
      The tests directory now contains a `README.md` file that describes how this
      works.
      
      This commit also includes additional cleanups and fixes:
      
      * Tests can now be tagged.
      * Tag-based selection is now unified across `start_server`, `tags` and `test`.
      * More information is provided about skipped or ignored tests.
      * Repeated patterns in tests have been extracted to common procedures, both at a
        global level and on a per-test file basis.
      * Cleaned up some cases where test setup was based on a previous test executing
        (a major anti-pattern that repeats itself in many places).
      * Cleaned up some cases where test teardown was not part of a test (in the
        future we should have dedicated teardown code that executes even when tests
        fail).
      * Fixed some tests that were flaky running on external servers.
      8a86bca5
  22. 30 May, 2021 1 commit
    • ny0312's avatar
      Always replicate time-to-live(TTL) as absolute timestamps in milliseconds (#8474) · 53d1acd5
      ny0312 authored
      Till now, on replica full-sync we used to transfer absolute time for TTL,
      however when a command arrived (EXPIRE or EXPIREAT),
      we used to propagate it as is to replicas (possibly with relative time),
      but always translate it to EXPIREAT (absolute time) to AOF.
      
      This commit changes that and will always use absolute time for propagation.
      see discussion in #8433
      
      Furthermore, we Introduce new commands: `EXPIRETIME/PEXPIRETIME`
      that allow extracting the absolute TTL time from a key.
      53d1acd5
  23. 20 May, 2021 1 commit
    • YaacovHazan's avatar
      stabilize tests that involved with load handlers (#8967) · 32a2584e
      YaacovHazan authored
      When test stop 'load handler' by killing the process that generating the load,
      some commands that already in the input buffer, still might be processed by the server.
      This may cause some instability in tests, that count on that no more commands
      processed after we stop the `load handler'
      
      In this commit, new proc 'wait_load_handlers_disconnected' added, to verify that no more
      cammands from any 'load handler' prossesed, by checking that the clients who
      genreate the load is disconnceted.
      
      Also, replacing check of dbsize with wait_for_ofs_sync before comparing debug digest, as
      it would fail in case the last key the workload wrote was an overridden key (not a new one).
      
      Affected tests
      Race fix:
      - failover command to specific replica works
      - Connect multiple replicas at the same time (issue #141), master diskless=$mdl, replica diskless=$sdl
      - AOF rewrite during write load: RDB preamble=$rdbpre
      
      Cleanup and speedup:
      - Test replication with blocking lists and sorted sets operations
      - Test replication with parallel clients writing in different DBs
      - Test replication partial resync: $descr (diskless: $mdl, $sdl, reconnect: $reconnect
      32a2584e
  24. 25 Apr, 2021 1 commit
  25. 19 Apr, 2021 1 commit
    • Hanna Fadida's avatar
      Modules: adding a module type for key space notification (#8759) · 53a4d6c3
      Hanna Fadida authored
      Adding a new type mask ​for key space notification, REDISMODULE_NOTIFY_MODULE, to enable unique notifications from commands on REDISMODULE_KEYTYPE_MODULE type keys (which is currently unsupported).
      
      Modules can subscribe to a module key keyspace notification by RM_SubscribeToKeyspaceEvents,
      and clients by notify-keyspace-events of redis.conf or via the CONFIG SET, with the characters 'd' or 'A' 
      (REDISMODULE_NOTIFY_MODULE type mask is part of the '**A**ll' notation for key space notifications).
      
      Refactor: move some pubsub test infra from pubsub.tcl to util.tcl to be re-used by other tests.
      53a4d6c3
  26. 18 Apr, 2021 1 commit
    • Oran Agra's avatar
      Improve testsuite print of log file (#8805) · f4b5a4d8
      Oran Agra authored
      1. the `dump_logs` option would have printed only logs of servers that were
         spawn before the test proc started, and not ones that the test proc
         started inside it.
      2. when a server proc catches an exception it should normally forward the
         exception upwards, specifically when it's an assertion that should be
         caught by a test proc above. however, in `durable` mode, we caught all
         exceptions printed them to stdout and let the code continue,
         this was wrong to do for assertions, which should have still been
         propagated to the test function.
      3. don't bother to search for crash log to print if we printed the the
         entire log anyway
      4. if no crash log was found, no need to print anything (i.e. the fact it
         wasn't found)
      5. rename warnings_from_file to crashlog_from_file
      f4b5a4d8
  27. 01 Apr, 2021 1 commit
    • sundb's avatar
      Use chi-square for random distributivity verification in test (#8709) · 569a3f45
      sundb authored
      Problem:
      Currently, when performing random distribution verification, we determine
      the probability of each element occurring in the sum, but the probability is
      only an estimate, these tests had rare sporadic failures, and we cannot verify
      what the probability of failure will be.
      
      Solution:
      Using the chi-square distribution instead of the original random distribution
      validation makes the test more reasonable and easier to find problems.
      569a3f45
  28. 30 Mar, 2021 1 commit
    • Viktor Söderqvist's avatar
      Add support for plaintext clients in TLS cluster (#8587) · 5629dbe7
      Viktor Söderqvist authored
      The cluster bus is established over TLS or non-TLS depending on the configuration tls-cluster. The client ports distributed in the cluster and sent to clients are assumed to be TLS or non-TLS also depending on tls-cluster.
      
      The cluster bus is now extended to also contain the non-TLS port of clients in a TLS cluster, when available. The non-TLS port of a cluster node, when available, is sent to clients connected without TLS in responses to CLUSTER SLOTS, CLUSTER NODES, CLUSTER SLAVES and MOVED and ASK redirects, instead of the TLS port.
      
      The user was able to override the client port by defining cluster-announce-port. Now cluster-announce-tls-port is added, so the user can define an alternative announce port for both TLS and non-TLS clients.
      
      Fixes #8134
      5629dbe7
  29. 29 Mar, 2021 1 commit
    • Sokolov Yura's avatar
      Add cluster slot migration tests (#8649) · 315df9ad
      Sokolov Yura authored
      
      
      Add tests for fixing migrating slot at all stages:
      
      1. when migration is half inited on "migrating" node
      2. when migration is half inited on "importing" node
      3. migration inited, but not finished
      4. migration is half finished on "migrating" node
      5. migration is half finished on "importing" node
      
      Also add tests for many simultaneous slot migrations.
      Co-authored-by: default avatarYossi Gottlieb <yossigo@gmail.com>
      315df9ad
  30. 16 Mar, 2021 1 commit
    • Huang Zhw's avatar
      When tests exit normally, some processes may still be alive (#8647) · a19c4058
      Huang Zhw authored
      In certain scenario start_server may think it failed to start a redis server
      although it started successfully. in these cases, it'll not terminate it, and
      it'll remain running when the test is over.
      
      In start_server if config doesn't have bind (the minimal.conf in introspection.tcl),
      it will try to bind ipv4 and ipv6. One may success while other fails. It will
      output "Could not create server TCP listening socket".
      wait_server_started uses this message to check whether instance started
      successfully. So it will consider that it failed even though redis started successfully.
      
      Additionally, in some cases it wasn't clear to users why the server exited,
      since the warning message printed to the log, could in some cases be harmless,
      and in some cases fatal.
      
      This PR adds makes a clear distinction between a warning log message and
      a fatal one, and changes the test suite to look for the fatal message.
      a19c4058
  31. 24 Feb, 2021 1 commit
  32. 21 Feb, 2021 1 commit
    • Huang Zw's avatar
      Client tracking tracking-redir-broken push len is 2 not 3 (#8456) · f687ac0c
      Huang Zw authored
      When redis responds with tracking-redir-broken push message (RESP3),
      it was responding with a broken protocol: an array of 3 elements, but only
      pushes 2 elements.
      
      Some bugs in the test make this pass. Read the push reply
      will consume an extra reply, because the reply length is 3, but there
      are only two elements, so the next reply will be treated as third
      element. So the test is corrected too.
      
      Other changes:
      * checkPrefixCollisionsOrReply success should return 1 instead of -1,
        this bug didn't have any implications.
      * improve client tracking tests to validate more of the response it reads.
      f687ac0c
  33. 07 Feb, 2021 1 commit
  34. 05 Feb, 2021 1 commit
    • sundb's avatar
      RAND* commands: fix risk of OOM panic in hash and zset, use fair random in... · 18ac4197
      sundb authored
      
      RAND* commands: fix risk of OOM panic in hash and zset, use fair random in hash, and add tests for even distribution to all (#8429)
      
      Changes to HRANDFIELD and ZRANDMEMBER:
      * Fix risk of OOM panic when client query a very big negative count (avoid allocating huge temporary buffer).
      * Fix uneven random distribution in HRANDFIELD with negative count (wasn't using dictGetFairRandomKey).
      * Add tests to check an even random distribution (HRANDFIELD, SRANDMEMBER, ZRANDMEMBER).
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      18ac4197
  35. 31 Jan, 2021 1 commit
    • Oran Agra's avatar
      Fix test issues from introduction of HRANDFIELD (#8424) · 5a7eb9c8
      Oran Agra authored
      * The corrupt dump fuzzer found a division by zero.
      * in some cases the random fields from the HRANDFIELD tests produced
        fields with newlines and other special chars (due to \ char), this caused
        the TCL tests to see a bulk response that has a newline in it and add {}
        around it, later it can think this is a nested list. in fact the `alpha` random
        string generator isn't using spaces and newlines, so it should not use `\`
        either.
      5a7eb9c8
  36. 29 Jan, 2021 1 commit
    • Yang Bodong's avatar
      Add HRANDFIELD and ZRANDMEMBER. improvements to SRANDMEMBER (#8297) · b9a0500f
      Yang Bodong authored
      
      
      New commands:
      `HRANDFIELD [<count> [WITHVALUES]]`
      `ZRANDMEMBER [<count> [WITHSCORES]]`
      Algorithms are similar to the one in SRANDMEMBER.
      
      Both return a simple bulk response when no arguments are given, and an array otherwise.
      In case values/scores are requested, RESP2 returns a long array, and RESP3 a nested array.
      note: in all 3 commands, the only option that also provides random order is the one with negative count.
      
      Changes to SRANDMEMBER
      * Optimization when count is 1, we can use the more efficient algorithm of non-unique random
      * optimization: work with sds strings rather than robj
      
      Other changes:
      * zzlGetScore: when zset needs to convert string to double, we use safer memcpy (in
        case the buffer is too small)
      * Solve a "bug" in SRANDMEMBER test: it intended to test a positive count (case 3 or
        case 4) and by accident used a negative count
      Co-authored-by: default avatarxinluton <xinluton@qq.com>
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      b9a0500f
  37. 28 Jan, 2021 1 commit
  38. 26 Jan, 2021 1 commit