1. 21 Jun, 2021 2 commits
  2. 20 Jun, 2021 4 commits
  3. 17 Jun, 2021 3 commits
    • gourav's avatar
    • sundb's avatar
      Make readQueryFromClient more aggressive when reading big arg again (Followup for #9003) (#9100) · 1a22445d
      sundb authored
      Due to the change in #9003, a long-standing bug was raised under `valgrind`.
      This bug can cause the master-slave sync to take a very long time, causing the `pendingquerybuf.tcl` test to fail.
      This problem does not only occur in master-slave sync, it is triggered when the big arg is greater than 32k.
      step:
      ```sh
      dd if=/dev/zero of=bigfile bs=1M count=32
      ./src/redis-cli -x hset a a < bigfile
      ```
      
      1) Make room for querybuf in processMultibulkBuffer, now the alloc of querybuf will be more than 32k.
      2) If this happens to trigger the `clientsCronResizeQueryBuffer`, querybuf will be resized to 0.
      3) Finally, in readQueryFromClient, we expand the querybuf non-greedily, from 0 to 32k.
          Old code, make room for querybuf is greedy, so it only needs 11 times to expand to 32M(16k*(2^11)),
          but now we need 2048(32*1024/16) times to reach it, due to the slow allocation under valgrind that exposed the problem.
      
      The fix for the excessive shrinking of the query buf to 0, will be handled in #5013 (that other change on it's own can fix failing test too), but the fix in this PR will also fix the failing test.
      
      The fix in this PR will makes the reading in `readQueryFromClient` more aggressive when working on a big arg (so that it is in par with the same code in `processMultibulkBuffer` (i.e. the two calls to `sdsMakeRoomForNonGreedy` should both use the bulk size).
      In the code before this fix the one in readQueryFromClient always has `readlen = PROTO_IOBUF_LEN`
      1a22445d
    • DarrenJiang13's avatar
      Improve objectComputeSize() with allocator fragmentaiton. (#9095) · ada60d80
      DarrenJiang13 authored
      This commit improve MEMORY USAGE command to include internal fragmentation overheads of:
      1. EMBSTR encoded strings
      2. ziplist encoded zsets and hashes
      3. List type nodes
      ada60d80
  4. 16 Jun, 2021 5 commits
    • Sam Bortman's avatar
      Support glob pattern matching for config include files (#8980) · c2b93ff8
      Sam Bortman authored
      This will allow distros to use an "include conf.d/*.conf" statement in the default configuration file
      which will facilitate customization across upgrades/downgrades.
      
      The change itself is trivial: instead of opening an individual file, the glob call creates a vector of files to open, and each file is opened in turn, and its content is added to the configuration.
      c2b93ff8
    • yoav-steinberg's avatar
      Remove gopher protocol support. (#9057) · 362786c5
      yoav-steinberg authored
      Gopher support was added mainly because it was simple (trivial to add).
      But apparently even something that was trivial at the time, does cause complications
      down the line when adding more features.
      We recently ran into a few issues with io-threads conflicting with the gopher support.
      We had to either complicate the code further in order to solve them, or drop gopher.
      AFAIK it's completely unused, so we wanna chuck it, rather than keep supporting it.
      362786c5
    • chenyang8094's avatar
      Enhance mem_usage/free_effort/unlink/copy callbacks and add GetDbFromIO api. (#8999) · e0cd3ad0
      chenyang8094 authored
      Create new module type enhanced callbacks: mem_usage2, free_effort2, unlink2, copy2.
      These will be given a context point from which the module can obtain the key name and database id.
      In addition the digest and defrag context can now be used to obtain the key name and database id.
      e0cd3ad0
    • Jason Elbaum's avatar
      Change return value type for ZPOPMAX/MIN in RESP3 (#8981) · 7f342020
      Jason Elbaum authored
      When using RESP3, ZPOPMAX/ZPOPMIN should return nested arrays for consistency
      with other commands (e.g. ZRANGE).
      
      We do that only when COUNT argument is present (similarly to how LPOP behaves).
      for reasoning see https://github.com/redis/redis/issues/8824#issuecomment-855427955
      
      This is a breaking change only when RESP3 is used, and COUNT argument is present!
      7f342020
    • Uri Shachar's avatar
      Cleaning up the cluster interface by moving almost all related declar… (#9080) · c7e502a0
      Uri Shachar authored
      * Cleaning up the cluster interface by moving almost all related declarations into cluster.h
      (no logic change -- just moving declarations/definitions around)
      
      This initial effort leaves two items out of scope - the configuration parsing into the server
      struct and the internals exposed by the clusterNode struct.
      
      * Remove unneeded declarations of dictSds*
      Ideally all the dictSds functionality would move from server.c into a dedicated module
      so we can avoid the duplication in redis-benchmark/cli
      
      * Move crc16 back into server.h, will be moved out once we create a seperate header file for
      hashing functions
      c7e502a0
  5. 15 Jun, 2021 5 commits
    • sundb's avatar
      Fix the wrong reisze of querybuf (#9003) · e5d8a5eb
      sundb authored
      The initialize memory of `querybuf` is `PROTO_IOBUF_LEN(1024*16) * 2` (due to sdsMakeRoomFor being greedy), under `jemalloc`, the allocated memory will be 40k.
      This will most likely result in the `querybuf` being resized when call `clientsCronResizeQueryBuffer` unless the client requests it fast enough.
      
      Note that this bug existed even before #7875, since the condition for resizing includes the sds headers (32k+6).
      
      ## Changes
      1. Use non-greedy sdsMakeRoomFor when allocating the initial query buffer (of 16k).
      1. Also use non-greedy allocation when working with BIG_ARG (we won't use that extra space anyway)
      2. in case we did use a greedy allocation, read as much as we can into the buffer we got (including internal frag), to reduce system calls.
      3. introduce a dedicated constant for the shrinking (same value as before)
      3. Add test for querybuf.
      4. improve a maxmemory test by ignoring the effect of replica query buffers (can accumulate many ACKs on slow env)
      5. improve a maxmemory by disabling slowlog (it will cause slight memory growth on slow env).
      e5d8a5eb
    • Binbin's avatar
      Check `CLIENT_DIRTY_CAS` flag before process transaction. (#9086) · eb15c456
      Binbin authored
      Do not queue command in an already aborted MULTI state.
      We can detect an error (watched key).
      So in queueMultiCommand, we also can return early.
      Like we deal with `CLIENT_DIRTY_EXEC`.
      eb15c456
    • DarrenJiang13's avatar
    • Binbin's avatar
      Fix XINFO help for unexpected options. (#9075) · b1099773
      Binbin authored
      Small cleanup and consistency.
      b1099773
    • yvette903's avatar
      Fix coredump after Client Unpause command when threaded I/O is enabled (#9041) · 096c5fd5
      yvette903 authored
      Fix crash when using io-threads-do-reads and issuing CLIENT PAUSE and
      CLIENT UNPAUSE.
      This issue was introduced in redis 6.2 together with the FAILOVER command.
      096c5fd5
  6. 14 Jun, 2021 5 commits
  7. 13 Jun, 2021 3 commits
    • Binbin's avatar
      Fix accidental deletion of sinterstore command when we meet wrong type error. (#9032) · b8a5da80
      Binbin authored
      SINTERSTORE would have deleted the dest key right away,
      even when later on it is bound to fail on an (WRONGTYPE) error.
      
      With this change it first picks up all the input keys, and only later
      delete the dest key if one is empty.
      
      Also add more tests for some commands.
      Mainly focus on 
      - `wrong type error`: 
      	expand test case (base on sinter bug) in non-store variant
      	add tests for store variant (although it exists in non-store variant, i think it would be better to have same tests)
      - the dstkey result when we meet `non-exist key (empty set)` in *store
      
      sdiff:
      - improve test case about wrong type error (the one we found in sinter, although it is safe in sdiff)
      - add test about using non-exist key (treat it like an empty set)
      sdiffstore:
      - according to sdiff test case, also add some tests about `wrong type error` and `non-exist key`
      - the different is that in sdiffstore, we will consider the `dstkey` result
      
      sunion/sunionstore add more tests (same as above)
      
      sinter/sinterstore also same as above ...
      b8a5da80
    • Binbin's avatar
      Remove duplicate dbid lookup in performEvictions. (#9063) · 5517f362
      Binbin authored
      Minor code cleanup.
      5517f362
    • ny0312's avatar
      Fix flaky test case for absolute TTL replication (#9069) · fb140a1b
      ny0312 authored
      The root cause is that one test (`5 keys in, 5 keys out`) is leaking a volatile key
      that can expire while another later test(`All TTL in commands are propagated
      as absolute timestamp in replication stream`) is running.
      Such leaked expiration injects an unexpected `DEL` command into the
      replication command during the later test, causing it to fail.
      
      The fixes are two fold:
      1. Plug the leak in the first test.
      2. Add FLUSHALL to the later test, to avoid future interference from other tests.
      fb140a1b
  8. 10 Jun, 2021 2 commits
    • ZhaolongLi's avatar
      63da66bb
    • Binbin's avatar
      Fixed some typos, add a spell check ci and others minor fix (#8890) · 0bfccc55
      Binbin authored
      This PR adds a spell checker CI action that will fail future PRs if they introduce typos and spelling mistakes.
      This spell checker is based on blacklist of common spelling mistakes, so it will not catch everything,
      but at least it is also unlikely to cause false positives.
      
      Besides that, the PR also fixes many spelling mistakes and types, not all are a result of the spell checker we use.
      
      Here's a summary of other changes:
      1. Scanned the entire source code and fixes all sorts of typos and spelling mistakes (including missing or extra spaces).
      2. Outdated function / variable / argument names in comments
      3. Fix outdated keyspace masks error log when we check `config.notify-keyspace-events` in loadServerConfigFromString.
      4. Trim the white space at the end of line in `module.c`. Check: https://github.com/redis/redis/pull/7751
      5. Some outdated https link URLs.
      6. Fix some outdated comment. Such as:
          - In README: about the rdb, we used to said create a `thread`, change to `process`
          - dbRandomKey function coment (about the dictGetRandomKey, change to dictGetFairRandomKey)
          - notifyKeyspaceEvent fucntion comment (add type arg)
          - Some others minor fix in comment (Most of them are incorrectly quoted by variable names)
      7. Modified the error log so that users can easily distinguish between TCP and TLS in `changeBindAddr`
      0bfccc55
  9. 08 Jun, 2021 5 commits
    • Wang Yuan's avatar
      Mem efficiency, make full use of client struct memory for reply buffers (#8968) · c396fd91
      Wang Yuan authored
      When we allocate a client struct with 16k reply buffer, the allocator we may give us 20K,
      This commit makes use of that extra space.
      Additionally, it tries to store whatever it can from the reply into the static 'buf' before
      allocating a new node for the reply list.
      c396fd91
    • Yang Bodong's avatar
      Verbose log enhancement: print client info when client exit (#9053) · 119121c7
      Yang Bodong authored
      It could be useful for debugging to know which client got disconnected.
      119121c7
    • guybe7's avatar
      Module API for current command name (#8792) · e16d3eb9
      guybe7 authored
      sometimes you can be very deep in the call stack, without access to argv.
      once you're there you may want your reply/log to contain the command name.
      e16d3eb9
    • ikeberlein's avatar
      Make redis-cli grep friendly in pubsub mode (#9013) · 77d44bb7
      ikeberlein authored
      redis-cli is grep friendly for all commands but SUBSCRIBE/PSUBSCRIBE.
      it is unable to process output from these commands line by line piped
      to another program because of output buffering. to overcome this
      situation I propose to flush stdout each time when it is written with
      reply from these commands the same way it is already done for all other
      commands.
      77d44bb7
    • Huang Zhw's avatar
      Fix some minor bugs in redis-cli (#8982) · 1df8c129
      Huang Zhw authored
      * clusterManagerAddSlots check argv_idx error.
      
      * clusterManagerLoadInfoFromNode remove unused param opts.
      
      * redis-cli node->ip may be an sds or a c string. Using %s to format
      is always right, %S may be wrong.
      
      * In clusterManagerFixOpenSlot clusterManagerBumpEpoch call is redundant,
      because it is already called in clusterManagerSetSlotOwner.
      
      * redis-cli cluster help add more commands in help messages.
      1df8c129
  10. 07 Jun, 2021 4 commits
  11. 06 Jun, 2021 2 commits