1. 16 Jun, 2021 3 commits
    • chenyang8094's avatar
      Enhance mem_usage/free_effort/unlink/copy callbacks and add GetDbFromIO api. (#8999) · e0cd3ad0
      chenyang8094 authored
      Create new module type enhanced callbacks: mem_usage2, free_effort2, unlink2, copy2.
      These will be given a context point from which the module can obtain the key name and database id.
      In addition the digest and defrag context can now be used to obtain the key name and database id.
      e0cd3ad0
    • Jason Elbaum's avatar
      Change return value type for ZPOPMAX/MIN in RESP3 (#8981) · 7f342020
      Jason Elbaum authored
      When using RESP3, ZPOPMAX/ZPOPMIN should return nested arrays for consistency
      with other commands (e.g. ZRANGE).
      
      We do that only when COUNT argument is present (similarly to how LPOP behaves).
      for reasoning see https://github.com/redis/redis/issues/8824#issuecomment-855427955
      
      This is a breaking change only when RESP3 is used, and COUNT argument is present!
      7f342020
    • Uri Shachar's avatar
      Cleaning up the cluster interface by moving almost all related declar… (#9080) · c7e502a0
      Uri Shachar authored
      * Cleaning up the cluster interface by moving almost all related declarations into cluster.h
      (no logic change -- just moving declarations/definitions around)
      
      This initial effort leaves two items out of scope - the configuration parsing into the server
      struct and the internals exposed by the clusterNode struct.
      
      * Remove unneeded declarations of dictSds*
      Ideally all the dictSds functionality would move from server.c into a dedicated module
      so we can avoid the duplication in redis-benchmark/cli
      
      * Move crc16 back into server.h, will be moved out once we create a seperate header file for
      hashing functions
      c7e502a0
  2. 15 Jun, 2021 5 commits
    • sundb's avatar
      Fix the wrong reisze of querybuf (#9003) · e5d8a5eb
      sundb authored
      The initialize memory of `querybuf` is `PROTO_IOBUF_LEN(1024*16) * 2` (due to sdsMakeRoomFor being greedy), under `jemalloc`, the allocated memory will be 40k.
      This will most likely result in the `querybuf` being resized when call `clientsCronResizeQueryBuffer` unless the client requests it fast enough.
      
      Note that this bug existed even before #7875, since the condition for resizing includes the sds headers (32k+6).
      
      ## Changes
      1. Use non-greedy sdsMakeRoomFor when allocating the initial query buffer (of 16k).
      1. Also use non-greedy allocation when working with BIG_ARG (we won't use that extra space anyway)
      2. in case we did use a greedy allocation, read as much as we can into the buffer we got (including internal frag), to reduce system calls.
      3. introduce a dedicated constant for the shrinking (same value as before)
      3. Add test for querybuf.
      4. improve a maxmemory test by ignoring the effect of replica query buffers (can accumulate many ACKs on slow env)
      5. improve a maxmemory by disabling slowlog (it will cause slight memory growth on slow env).
      e5d8a5eb
    • Binbin's avatar
      Check `CLIENT_DIRTY_CAS` flag before process transaction. (#9086) · eb15c456
      Binbin authored
      Do not queue command in an already aborted MULTI state.
      We can detect an error (watched key).
      So in queueMultiCommand, we also can return early.
      Like we deal with `CLIENT_DIRTY_EXEC`.
      eb15c456
    • DarrenJiang13's avatar
    • Binbin's avatar
      Fix XINFO help for unexpected options. (#9075) · b1099773
      Binbin authored
      Small cleanup and consistency.
      b1099773
    • yvette903's avatar
      Fix coredump after Client Unpause command when threaded I/O is enabled (#9041) · 096c5fd5
      yvette903 authored
      Fix crash when using io-threads-do-reads and issuing CLIENT PAUSE and
      CLIENT UNPAUSE.
      This issue was introduced in redis 6.2 together with the FAILOVER command.
      096c5fd5
  3. 14 Jun, 2021 5 commits
  4. 13 Jun, 2021 3 commits
    • Binbin's avatar
      Fix accidental deletion of sinterstore command when we meet wrong type error. (#9032) · b8a5da80
      Binbin authored
      SINTERSTORE would have deleted the dest key right away,
      even when later on it is bound to fail on an (WRONGTYPE) error.
      
      With this change it first picks up all the input keys, and only later
      delete the dest key if one is empty.
      
      Also add more tests for some commands.
      Mainly focus on 
      - `wrong type error`: 
      	expand test case (base on sinter bug) in non-store variant
      	add tests for store variant (although it exists in non-store variant, i think it would be better to have same tests)
      - the dstkey result when we meet `non-exist key (empty set)` in *store
      
      sdiff:
      - improve test case about wrong type error (the one we found in sinter, although it is safe in sdiff)
      - add test about using non-exist key (treat it like an empty set)
      sdiffstore:
      - according to sdiff test case, also add some tests about `wrong type error` and `non-exist key`
      - the different is that in sdiffstore, we will consider the `dstkey` result
      
      sunion/sunionstore add more tests (same as above)
      
      sinter/sinterstore also same as above ...
      b8a5da80
    • Binbin's avatar
      Remove duplicate dbid lookup in performEvictions. (#9063) · 5517f362
      Binbin authored
      Minor code cleanup.
      5517f362
    • ny0312's avatar
      Fix flaky test case for absolute TTL replication (#9069) · fb140a1b
      ny0312 authored
      The root cause is that one test (`5 keys in, 5 keys out`) is leaking a volatile key
      that can expire while another later test(`All TTL in commands are propagated
      as absolute timestamp in replication stream`) is running.
      Such leaked expiration injects an unexpected `DEL` command into the
      replication command during the later test, causing it to fail.
      
      The fixes are two fold:
      1. Plug the leak in the first test.
      2. Add FLUSHALL to the later test, to avoid future interference from other tests.
      fb140a1b
  5. 10 Jun, 2021 2 commits
    • ZhaolongLi's avatar
      63da66bb
    • Binbin's avatar
      Fixed some typos, add a spell check ci and others minor fix (#8890) · 0bfccc55
      Binbin authored
      This PR adds a spell checker CI action that will fail future PRs if they introduce typos and spelling mistakes.
      This spell checker is based on blacklist of common spelling mistakes, so it will not catch everything,
      but at least it is also unlikely to cause false positives.
      
      Besides that, the PR also fixes many spelling mistakes and types, not all are a result of the spell checker we use.
      
      Here's a summary of other changes:
      1. Scanned the entire source code and fixes all sorts of typos and spelling mistakes (including missing or extra spaces).
      2. Outdated function / variable / argument names in comments
      3. Fix outdated keyspace masks error log when we check `config.notify-keyspace-events` in loadServerConfigFromString.
      4. Trim the white space at the end of line in `module.c`. Check: https://github.com/redis/redis/pull/7751
      5. Some outdated https link URLs.
      6. Fix some outdated comment. Such as:
          - In README: about the rdb, we used to said create a `thread`, change to `process`
          - dbRandomKey function coment (about the dictGetRandomKey, change to dictGetFairRandomKey)
          - notifyKeyspaceEvent fucntion comment (add type arg)
          - Some others minor fix in comment (Most of them are incorrectly quoted by variable names)
      7. Modified the error log so that users can easily distinguish between TCP and TLS in `changeBindAddr`
      0bfccc55
  6. 09 Jun, 2021 1 commit
    • Yossi Gottlieb's avatar
      Improve test suite to handle external servers better. (#9033) · 8a86bca5
      Yossi Gottlieb authored
      This commit revives the improves the ability to run the test suite against
      external servers, instead of launching and managing `redis-server` processes as
      part of the test fixture.
      
      This capability existed in the past, using the `--host` and `--port` options.
      However, it was quite limited and mostly useful when running a specific tests.
      Attempting to run larger chunks of the test suite experienced many issues:
      
      * Many tests depend on being able to start and control `redis-server` themselves,
      and there's no clear distinction between external server compatible and other
      tests.
      * Cluster mode is not supported (resulting with `CROSSSLOT` errors).
      
      This PR cleans up many things and makes it possible to run the entire test suite
      against an external server. It also provides more fine grained controls to
      handle cases where the external server supports a subset of the Redis commands,
      limited number of databases, cluster mode, etc.
      
      The tests directory now contains a `README.md` file that describes how this
      works.
      
      This commit also includes additional cleanups and fixes:
      
      * Tests can now be tagged.
      * Tag-based selection is now unified across `start_server`, `tags` and `test`.
      * More information is provided about skipped or ignored tests.
      * Repeated patterns in tests have been extracted to common procedures, both at a
        global level and on a per-test file basis.
      * Cleaned up some cases where test setup was based on a previous test executing
        (a major anti-pattern that repeats itself in many places).
      * Cleaned up some cases where test teardown was not part of a test (in the
        future we should have dedicated teardown code that executes even when tests
        fail).
      * Fixed some tests that were flaky running on external servers.
      8a86bca5
  7. 08 Jun, 2021 5 commits
    • Wang Yuan's avatar
      Mem efficiency, make full use of client struct memory for reply buffers (#8968) · c396fd91
      Wang Yuan authored
      When we allocate a client struct with 16k reply buffer, the allocator we may give us 20K,
      This commit makes use of that extra space.
      Additionally, it tries to store whatever it can from the reply into the static 'buf' before
      allocating a new node for the reply list.
      c396fd91
    • Yang Bodong's avatar
      Verbose log enhancement: print client info when client exit (#9053) · 119121c7
      Yang Bodong authored
      It could be useful for debugging to know which client got disconnected.
      119121c7
    • guybe7's avatar
      Module API for current command name (#8792) · e16d3eb9
      guybe7 authored
      sometimes you can be very deep in the call stack, without access to argv.
      once you're there you may want your reply/log to contain the command name.
      e16d3eb9
    • ikeberlein's avatar
      Make redis-cli grep friendly in pubsub mode (#9013) · 77d44bb7
      ikeberlein authored
      redis-cli is grep friendly for all commands but SUBSCRIBE/PSUBSCRIBE.
      it is unable to process output from these commands line by line piped
      to another program because of output buffering. to overcome this
      situation I propose to flush stdout each time when it is written with
      reply from these commands the same way it is already done for all other
      commands.
      77d44bb7
    • Huang Zhw's avatar
      Fix some minor bugs in redis-cli (#8982) · 1df8c129
      Huang Zhw authored
      * clusterManagerAddSlots check argv_idx error.
      
      * clusterManagerLoadInfoFromNode remove unused param opts.
      
      * redis-cli node->ip may be an sds or a c string. Using %s to format
      is always right, %S may be wrong.
      
      * In clusterManagerFixOpenSlot clusterManagerBumpEpoch call is redundant,
      because it is already called in clusterManagerSetSlotOwner.
      
      * redis-cli cluster help add more commands in help messages.
      1df8c129
  8. 07 Jun, 2021 4 commits
  9. 06 Jun, 2021 3 commits
    • Oran Agra's avatar
      tests: prevent name clash in variables leading to wrong test name (#8995) · 3e39ea0b
      Oran Agra authored
      running the "geo" unit would have shown that it completed a unit named
      "north". this was because the variable `$name` was overwritten.
      This commit isn't perfect, but it slightly reduces the chance for
      variable name clash.
      
      ```
      $ ./runtest --single unit/geo
      .......
      Testing unit/geo
      .......
      [1/1 done]: north (15 seconds)
      ```
      3e39ea0b
    • Huang Zhw's avatar
      XTRIM call streamParseAddOrTrimArgsOrReply use wrong arg xadd. (#9047) · 91f3689b
      Huang Zhw authored
      xtrimCommand call streamParseAddOrTrimArgsOrReply should use xadd==0.
      
      When the syntax is valid, it does not cause any bugs because the params of XADD is superset of XTRIM.
      Just XTRIM will not respond with error on invalid syntax. The syntax of XADD will also be accpeted by XTRIM.
      91f3689b
    • Binbin's avatar
      Add missing zuiClearIterator in zrandmemberWithCountCommand. (#8979) · bb671d90
      Binbin authored
      Also the bug that currently does not cause memory leaks.
      Because op->type = OBJ_ZSET, in zuiClearIterator, we will do nothing.
      Just a cleanup that zuiInitIterator and zuiClearIterator should appear in pairs.
      bb671d90
  10. 04 Jun, 2021 1 commit
  11. 03 Jun, 2021 2 commits
  12. 02 Jun, 2021 2 commits
  13. 01 Jun, 2021 4 commits