1. 11 Jan, 2023 1 commit
    • Viktor Söderqvist's avatar
      Make dictEntry opaque · c84248b5
      Viktor Söderqvist authored
      Use functions for all accesses to dictEntry (except in dict.c). Dict abuses
      e.g. in defrag.c have been replaced by support functions provided by dict.
      c84248b5
  2. 10 Jan, 2023 3 commits
  3. 08 Jan, 2023 1 commit
  4. 05 Jan, 2023 3 commits
  5. 04 Jan, 2023 4 commits
    • Binbin's avatar
      Make redis-cli support PSYNC command (#11647) · 4ef4c4a6
      Binbin authored
      
      
      The current redis-cli does not support the real PSYNC command, the older
      version of redis-cli can support PSYNC is because that we actually issue
      the SYNC command instead of PSYNC, so it act like SYNC (always full-sync).
      Noted that in this case we will send the SYNC first (triggered by sendSync),
      then send the PSYNC (the one in redis-cli input).
      
      Didn't bother to find which version that the order changed, we send PSYNC
      first (the one in redis-cli input), and then send the SYNC (the one triggered
      by sendSync). So even full-sync is not working anymore, and it will result
      this output (mentioned in issue #11246):
      ```
      psync dummy 0
      Entering replica output mode...  (press Ctrl-C to quit)
      SYNC with master, discarding bytes of bulk transfer until EOF marker...
      Error reading RDB payload while SYNCing
      ```
      
      This PR adds PSYNC support to redis-cli, which can handle +FULLRESYNC and
      +CONTINUE responses, and some examples will follow.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      4ef4c4a6
    • Oran Agra's avatar
      Fix potential issue with Lua argv caching, module command filter and libc realloc (#11652) · c8052122
      Oran Agra authored
      TLDR: solve a problem introduced in Redis 7.0.6 (#11541) with
      RM_CommandFilterArgInsert being called from scripts, which can
      lead to memory corruption.
      
      Libc realloc can return the same pointer even if the size was changed. The code in
      freeLuaRedisArgv had an assumption that if the pointer didn't change, then the
      allocation didn't change, and the cache can still be reused.
      However, if rewriteClientCommandArgument or RM_CommandFilterArgInsert were
      used, it could be that we realloced the argv array, and the pointer didn't change, then
      a consecutive command being executed from Lua can use that argv cache reaching
      beyond its size.
      This was actually only possible with modules, since the decision to realloc was based
      on argc, rather than argv_len.
      c8052122
    • Oran Agra's avatar
      fix handshake timeout replication test race (#11640) · 0ecf6cdc
      Oran Agra authored
      Test on ARM + TLS often fail with this error:
      ```
      *** [err]: Slave is able to detect timeout during handshake in tests/integration/replication.tcl
      Replica is not able to detect timeout
      ```
      https://github.com/redis/redis-extra-ci/actions/runs/3727554226/jobs/6321797837
      
      The replica logs show that in this case the replica got timeout before even getting a response to the PING command (instead of the SYNC command).
      
      it should have shown these:
      ```
      * MASTER <-> REPLICA sync started
      * REPLICAOF 127.0.0.1:22112 enabled ....
      ### Starting test Slave enters handshake in tests/integration/replication.tcl
      * Non blocking connect for SYNC fired the event.
      ```
      then:
      ```
      * Master replied to PING, replication can continue...
      * Trying a partial resynchronization (request 50da9eff70d774f4e6cb723eb4b091440f215772:1).
      ```
      and then hang for 5 seconds:
      ```
      # Timeout connecting to the MASTER...
      * Reconnecting to MASTER 127.0.0.1:21112 after failure
      ```
      
      but instead it got this (looks like it disconnected too early, and then tried to re-connect):
      ```
      10890:M 19 Dec 2022 01:32:54.794 * Ready to accept connections tls
      10890:M 19 Dec 2022 01:32:54.809 - Accepted 127.0.0.1:41047
      10890:M 19 Dec 2022 01:32:54.878 - Reading from client: error:0A000126:SSL routines::unexpected eof while reading
      10890:M 19 Dec 2022 01:32:54.925 - Accepted 127.0.0.1:39207
      10890:S 19 Dec 2022 01:32:55.463 * Before turning into a replica, using my own master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
      10890:S 19 Dec 2022 01:32:55.463 * Connecting to MASTER 127.0.0.1:24126
      10890:S 19 Dec 2022 01:32:55.463 * MASTER <-> REPLICA sync started
      10890:S 19 Dec 2022 01:32:55.463 * REPLICAOF 127.0.0.1:24126 enabled (user request from 'id=4 addr=127.0.0.1:39207 laddr=127.0.0.1:24125 fd=8 name= age=1 idle=0 flags=N db=9 sub=0 psub=0 ssub=0 multi=-1 qbuf=43 qbuf-free=20431 argv-mem=21 multi-mem=0 rbs=1024 rbp=5 obl=0 oll=0 omem=0 tot-mem=22317 events=r cmd=slaveof user=default redir=-1 resp=2')
      ### Starting test Slave enters handshake in tests/integration/replication.tcl
      10890:S 19 Dec 2022 01:32:55.476 * Non blocking connect for SYNC fired the event.
      10890:S 19 Dec 2022 01:33:00.701 # Failed to read response from the server: (null)         <- note this!!
      10890:S 19 Dec 2022 01:33:00.701 # Master did not respond to command during SYNC handshake
      10890:S 19 Dec 2022 01:33:01.002 * Connecting to MASTER 127.0.0.1:24126
      10890:S 19 Dec 2022 01:33:01.002 * MASTER <-> REPLICA sync started
      ### Starting test Slave is able to detect timeout during handshake in tests/integration/replication.tcl
      10890:S 19 Dec 2022 01:33:05.497 * Non blocking connect for SYNC fired the event.
      10890:S 19 Dec 2022 01:33:05.500 * Master replied to PING, replication can continue...
      10890:S 19 Dec 2022 01:33:05.510 * Trying a partial resynchronization (request 947e1956372a0e6c819cfec51c42cc7979b0c221:1).
      10890:S 19 Dec 2022 01:34:05.833 # Failed to read response from the server: error:0A000126:SSL routines::unexpected eof while reading
      10890:S 19 Dec 2022 01:34:05.833 # Master did not reply to PSYNC, will try later
      ```
      
      This PR sets enables the 5 seconds timeout at a later stage to try and prevent the early disconnection.
      0ecf6cdc
    • zhenwei pi's avatar
      Introduce .is_local method for connection layer (#11672) · dec529f4
      zhenwei pi authored
      
      
      Introduce .is_local method to connection, and implement for TCP/TLS/
      Unix socket, also drop 'int islocalClient(client *c)'. Then we can
      hide the detail into the specific connection types.
      Uplayer tests a connection is local or not by abstract method only.
      Signed-off-by: default avatarzhenwei pi <pizhenwei@bytedance.com>
      Signed-off-by: default avatarzhenwei pi <pizhenwei@bytedance.com>
      dec529f4
  6. 03 Jan, 2023 1 commit
  7. 02 Jan, 2023 1 commit
  8. 01 Jan, 2023 1 commit
    • ranshid's avatar
      reprocess command when client is unblocked on keys (#11012) · 383d902c
      ranshid authored
      *TL;DR*
      ---------------------------------------
      Following the discussion over the issue [#7551](https://github.com/redis/redis/issues/7551
      
      )
      We decided to refactor the client blocking code to eliminate some of the code duplications
      and to rebuild the infrastructure better for future key blocking cases.
      
      
      *In this PR*
      ---------------------------------------
      1. reprocess the command once a client becomes unblocked on key (instead of running
         custom code for the unblocked path that's different than the one that would have run if
         blocking wasn't needed)
      2. eliminate some (now) irrelevant code for handling unblocking lists/zsets/streams etc...
      3. modify some tests to intercept the error in cases of error on reprocess after unblock (see
         details in the notes section below)
      4. replace '$' on the client argv with current stream id. Since once we reprocess the stream
         XREAD we need to read from the last msg and not wait for new msg  in order to prevent
         endless block loop. 
      5. Added statistics to the info "Clients" section to report the:
         * `total_blocking_keys` - number of blocking keys
         * `total_blocking_keys_on_nokey` - number of blocking keys which have at least 1 client
            which would like
         to be unblocked on when the key is deleted.
      6. Avoid expiring unblocked key during unblock. Previously we used to lookup the unblocked key
         which might have been expired during the lookup. Now we lookup the key using NOTOUCH and
         NOEXPIRE to avoid deleting it at this point, so propagating commands in blocked.c is no longer needed.
      7. deprecated command flags. We decided to remove the CMD_CALL_STATS and CMD_CALL_SLOWLOG
         and make an explicit verification in the call() function in order to decide if stats update should take place.
         This should simplify the logic and also mitigate existing issues: for example module calls which are
         triggered as part of AOF loading might still report stats even though they are called during AOF loading.
      
      *Behavior changes*
      ---------------------------------------------------
      
      1. As this implementation prevents writing dedicated code handling unblocked streams/lists/zsets,
      since we now re-process the command once the client is unblocked some errors will be reported differently.
      The old implementation used to issue
      ``UNBLOCKED the stream key no longer exists``
      in the following cases:
         - The stream key has been deleted (ie. calling DEL)
         - The stream and group existed but the key type was changed by overriding it (ie. with set command)
         - The key not longer exists after we swapdb with a db which does not contains this key
         - After swapdb when the new db has this key but with different type.
         
      In the new implementation the reported errors will be the same as if the command was processed after effect:
      **NOGROUP** - in case key no longer exists, or **WRONGTYPE** in case the key was overridden with a different type.
      
      2. Reprocessing the command means that some checks will be reevaluated once the
      client is unblocked.
      For example, ACL rules might change since the command originally was executed and
      will fail once the client is unblocked.
      Another example is OOM condition checks which might enable the command to run and
      block but fail the command reprocess once the client is unblocked.
      
      3. One of the changes in this PR is that no command stats are being updated once the
      command is blocked (all stats will be updated once the client is unblocked). This implies
      that when we have many clients blocked, users will no longer be able to get that information
      from the command stats. However the information can still be gathered from the client list.
      
      **Client blocking**
      ---------------------------------------------------
      
      the blocking on key will still be triggered the same way as it is done today.
      in order to block the current client on list of keys, the call to
      blockForKeys will still need to be made which will perform the same as it is today:
      
      *  add the client to the list of blocked clients on each key
      *  keep the key with a matching list node (position in the global blocking clients list for that key)
         in the client private blocking key dict.
      *  flag the client with CLIENT_BLOCKED
      *  update blocking statistics
      *  register the client on the timeout table
      
      **Key Unblock**
      ---------------------------------------------------
      
      Unblocking a specific key will be triggered (same as today) by calling signalKeyAsReady.
      the implementation in that part will stay the same as today - adding the key to the global readyList.
      The reason to maintain the readyList (as apposed to iterating over all clients blocked on the specific key)
      is in order to keep the signal operation as short as possible, since it is called during the command processing.
      The main change is that instead of going through a dedicated code path that operates the blocked command
      we will just call processPendingCommandsAndResetClient.
      
      **ClientUnblock (keys)**
      ---------------------------------------------------
      
      1. Unblocking clients on keys will be triggered after command is
         processed and during the beforeSleep
      8. the general schema is:
      9. For each key *k* in the readyList:
      ```            
      For each client *c* which is blocked on *k*:
                  in case either:
      	          1. *k* exists AND the *k* type matches the current client blocking type
      	  	      OR
      	          2. *k* exists and *c* is blocked on module command
      	    	      OR
      	          3. *k* does not exists and *c* was blocked with the flag
      	             unblock_on_deleted_key
                       do:
                                        1. remove the client from the list of clients blocked on this key
                                        2. remove the blocking list node from the client blocking key dict
                                        3. remove the client from the timeout list
                                        10. queue the client on the unblocked_clients list
                                        11. *NEW*: call processCommandAndResetClient(c);
      ```
      *NOTE:* for module blocked clients we will still call the moduleUnblockClientByHandle
                    which will queue the client for processing in moduleUnblockedClients list.
      
      **Process Unblocked clients**
      ---------------------------------------------------
      
      The process of all unblocked clients is done in the beforeSleep and no change is planned
      in that part.
      
      The general schema will be:
      For each client *c* in server.unblocked_clients:
      
              * remove client from the server.unblocked_clients
              * set back the client readHandler
              * continue processing the pending command and input buffer.
      
      *Some notes regarding the new implementation*
      ---------------------------------------------------
      
      1. Although it was proposed, it is currently difficult to remove the
         read handler from the client while it is blocked.
         The reason is that a blocked client should be unblocked when it is
         disconnected, or we might consume data into void.
      
      2. While this PR mainly keep the current blocking logic as-is, there
         might be some future additions to the infrastructure that we would
         like to have:
         - allow non-preemptive blocking of client - sometimes we can think
           that a new kind of blocking can be expected to not be preempt. for
           example lets imagine we hold some keys on disk and when a command
           needs to process them it will block until the keys are uploaded.
           in this case we will want the client to not disconnect or be
           unblocked until the process is completed (remove the client read
           handler, prevent client timeout, disable unblock via debug command etc...).
         - allow generic blocking based on command declared keys - we might
           want to add a hook before command processing to check if any of the
           declared keys require the command to block. this way it would be
           easier to add new kinds of key-based blocking mechanisms.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      Signed-off-by: default avatarRan Shidlansik <ranshid@amazon.com>
      383d902c
  9. 28 Dec, 2022 1 commit
    • sundb's avatar
      Remove unnecessary updateClientMemUsageAndBucket() when feeding monitors (#11657) · af0a4fe2
      sundb authored
      This call is introduced in #8687, but became irrelevant in #11348, and is currently a no-op.
      The fact is that #11348 an unintended side effect, which is that even if the client eviction config
      is enabled, there are certain types of clients for which memory consumption is not accurately
      tracked, and so unlike normal clients, their memory isn't reported correctly in INFO.
      af0a4fe2
  10. 23 Dec, 2022 1 commit
    • Madelyn Olson's avatar
      Harden init-tests for cluster tests (#11635) · 7379d221
      Madelyn Olson authored
      Attempt to harden cluster init-tests by doing two things:
      * Retry up to 3 times to join the cluster. Cluster meet is entirely idempotent, so it should stabilize if we missed a node.
      * Validate the connection is actually established, not just exists in the cluster list. Nodes can exist in handshake, but might later get dropped.
      7379d221
  11. 22 Dec, 2022 2 commits
    • Binbin's avatar
      Fix race in PSYNC2 partial resync test (#11653) · 9e1a00d6
      Binbin authored
      This test sometimes fails:
      ```
      *** [err]: PSYNC2: Partial resync after Master restart using RDB aux fields with expire in tests/integration/psync2-master-restart.tcl
      Expected [status ::redis::redisHandle24 sync_partial_ok] == 1 (context: type eval line 49 cmd {assert {[status $replica sync_partial_ok] == 1}} proc ::test)
      ```
      
      This is because the default repl-timeout value is 10s, sometimes the test
      got timeout, then it will do a reconnect, it will incr the sync_partial_ok
      counter, and then cause the test to fail.In this fix, we set the repl-timeout
      to a very large number to make sure we won't get the timeout.
      9e1a00d6
    • Binbin's avatar
      Fix flaky PTTL time to live in milliseconds test on slow machines (#11651) · 9b20d598
      Binbin authored
      This test failed in FreeBSD:
      ```
      *** [err]: PTTL returns time to live in milliseconds in tests/unit/expire.tcl
      Expected 836 > 900 && 836 <= 1000 (context: type eval line 5 cmd {assert {$ttl > 900 && $ttl <= 1000}} proc ::test)
      ```
      
      On some slow machines, sometimes the test take close to 200ms
      to finish. We only set aside 100ms, so that caused the failure.
      Since the failure was around 800, change the condition to be >500.
      9b20d598
  12. 20 Dec, 2022 1 commit
    • guybe7's avatar
      Cleanup: Get rid of server.core_propagates (#11572) · 9c7c6924
      guybe7 authored
      1. Get rid of server.core_propagates - we can just rely on module/call nesting levels
      2. Rename in_nested_call  to execution_nesting and update the comment
      3. Remove module_ctx_nesting (redundant, we can use execution_nesting)
      4. Modify postExecutionUnitOperations according to the comment (The main purpose of this PR)
      5. trackingHandlePendingKeyInvalidations: Check the nesting level inside this function
      9c7c6924
  13. 18 Dec, 2022 2 commits
    • Oran Agra's avatar
      fix race in list test with blocking commands (#11627) · 669688a3
      Oran Agra authored
      I've seen the `BRPOPLPUSH with multiple blocked clients` test hang.
      this probably happened because rd2 blocked before rd1 and then it was
      also released first, and rd1 remained blocked.
      
      ```
              r del blist{t} target1{t} target2{t}
              r set target1{t} nolist
              $rd1 brpoplpush blist{t} target1{t} 0
              $rd2 brpoplpush blist{t} target2{t} 0
              r lpush blist{t} foo
      
              assert_error "WRONGTYPE*" {$rd1 read}
              assert_equal {foo} [$rd2 read]
              assert_equal {foo} [r lrange target2{t} 0 -1]
      ```
      changes:
      * added all missing calls for wait_for_blocked_client after issuing blocking commands)
      * removed some excessive `after 100`
      * fix undetected crossslot error in BRPOPLPUSH test
      * rollback changes to proto-max-bulk-len so external tests can be rerun
      669688a3
    • Oran Agra's avatar
      fix flaky latency test (#11636) · 60f7111b
      Oran Agra authored
      Fix a flaky test that probably fails on overload timing issues.
      
      This unit starts with
      ```
          # Set a threshold high enough to avoid spurious latency events.
          r config set latency-monitor-threshold 200
      ```
      
      but later the test measuring expire event changes the threshold.
      this fix is to revert it to 200 after that test.
      
      Got this error (ARM+TLS)
      ```
      *** [err]: LATENCY RESET is able to reset events in tests/unit/latency-monitor.tcl
      Expected [r latency latest] eq {} (context: type eval line 3 cmd {assert {[r latency latest] eq {}}} proc ::test)
      ```
      60f7111b
  14. 16 Dec, 2022 1 commit
  15. 15 Dec, 2022 2 commits
  16. 13 Dec, 2022 1 commit
    • Binbin's avatar
      Fix races in swapdb async_loading test (#11613) · 5f69ce0d
      Binbin authored
      
      
      There is a race in the test:
      ```
      *** [err]: Diskless load swapdb (async_loading): new database is exposed after swapping in tests/integration/replication.tcl
      Expected 'myvalue' to be equal to '' (context: type eval line 3 cmd {assert_equal [$replica GET mykey] ""} proc ::test)
      ```
      
      When doing `$replica GET mykey`, the replica is using the old database.
      The reason may be that when doing `master client kill type replica`,
      the replica did not yet realize it got disconnected from the master.
      So the check of master_link_status fails, and the replica did not
      finish the swapdb and the loading.
      
      In that case, i think the solution is to check the sync_full stat on
      the master and wait for it to get incremented from the previous value.
      i.e. the way to know that we're done with the full sync is not to check
      that our state is up (could be up if we check too early), but rather
      check that the sync_full counter got incremented.
      
      During the reviewing, we found another race, in Aborted testType,
      the `$master config set rdb-key-save-delay 10000` is done after we
      already initiated the disconnection, so there's a chance that the replica
      will attempt to reconnect before that call, in which case if we fork() before
      it, the config will not take effect. Move it to above the disconnection.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      5f69ce0d
  17. 12 Dec, 2022 2 commits
    • Oran Agra's avatar
      solve race in replication test due to ping (#11609) · cd12cc2f
      Oran Agra authored
      attach_to_replication_stream already stops pings, but it stops them on
      the server we connect to, and in this case it's a replica, and we need
      to stop them on the real master.
      cd12cc2f
    • Binbin's avatar
      Fix timing issue in replication test (#11611) · ef282bd7
      Binbin authored
      There is a timing issue in the test, happens with valgrind:
      ```
      *** [err]: diskless fast replicas drop during rdb pipe in tests/integration/replication.tcl
      log message of '"*Loading DB in memory*"' not found in ./tests/tmp/server.3580.246/stdout after line: 0 till line: 39
      ```
      
      The server logs:
      ```
      43465:S 03 Dec 2022 01:26:25.664 * Trying a partial resynchronization (request 15155fa24af0539b70428f9b41f4f7129d774560:1).
      43465:S 03 Dec 2022 01:26:35.133 * Full resync from master: 8ddf5a3f7c8ca1061c6b29aa84e7c985c5b29c61:680
      ```
      
      From the logs, we can see it took almost 10s to get full resync response,
      happens with valgrind. it's extremely slow. So i guess it's just an
      insufficient wait_for_condition timeout.
      
      Set the time to 15s, and modify other similar places at the same time.
      ef282bd7
  18. 11 Dec, 2022 1 commit
  19. 09 Dec, 2022 3 commits
    • Binbin's avatar
      Fix zuiFind crash / RM_ScanKey hang on SET object listpack encoding (#11581) · 20854cb6
      Binbin authored
      
      
      In #11290, we added listpack encoding for SET object.
      But forgot to support it in zuiFind, causes ZINTER, ZINTERSTORE,
      ZINTERCARD, ZIDFF, ZDIFFSTORE to crash.
      And forgot to support it in RM_ScanKey, causes it hang.
      
      This PR add support SET listpack in zuiFind, and in RM_ScanKey.
      And add tests for related commands to cover this case.
      
      Other changes:
      - There is no reason for zuiFind to go into the internals of the SET.
        It can simply use setTypeIsMember and don't care about encoding.
      - Remove the `#include "intset.h"` from server.h reduce the chance of
        accidental intset API use.
      - Move setTypeAddAux, setTypeRemoveAux and setTypeIsMemberAux
        interfaces to the header.
      - In scanGenericCommand, use setTypeInitIterator and setTypeNext
        to handle OBJ_SET scan.
      - In RM_ScanKey, improve hash scan mode, use lpGetValue like zset,
        they can share code and better performance.
      
      The zuiFind part fixes #11578
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      Co-authored-by: default avatarViktor Söderqvist <viktor.soderqvist@est.tech>
      20854cb6
    • Oran Agra's avatar
      Solve issues with active defrag test failing on fast machines (#11598) · 528bb11d
      Oran Agra authored
      We do defrag during AOF loading, but aim to detect fragmentation only
      once a second, so this test aims to slow down the AOF loading and mimic
      loading of a large file.
      On fast machines the sleep, plus the actual work we did was insufficient
      making it sleep longer so the test won't fail.
      
      The error we used to get is this one:
      Expected 0 > 100000 (context: type eval line 106 cmd {assert {$hits > 100000}} proc ::test)
      528bb11d
    • filipe oliveira's avatar
      Reduce rewriteClientCommandVector usage on EXPIRE command (#11602) · c3fb48da
      filipe oliveira authored
      
      
      There is overhead on Redis 7.0 EXPIRE command that is not present on 6.2.7. 
      
      We could see that on the unstable profile there are around 7% of CPU cycles
      spent on rewriteClientCommandVector that are not present on 6.2.7.
      This was introduced in #8474.
      This PR reduces the overhead by using 2X rewriteClientCommandArgument instead of
      rewriteClientCommandVector. In this scenario rewriteClientCommandVector creates 4 arguments.
      the above usage of rewriteClientCommandArgument reduces the overhead in half.
      
      This PR should also improve PEXPIREAT performance by avoiding at all
      rewriteClientCommandArgument usage. 
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      c3fb48da
  20. 08 Dec, 2022 3 commits
  21. 07 Dec, 2022 2 commits
    • CatboxParadox's avatar
      Use SNI on outgoing TLS connections (#11458) · 049f5d87
      CatboxParadox authored
      When establishing an outgoing TLS connection using a hostname as a target, use TLS SNI extensions to include the hostname in use.
      049f5d87
    • Harkrishn Patro's avatar
      Optimize client memory usage tracking operation while client eviction is disabled (#11348) · c0267b3f
      Harkrishn Patro authored
      
      
      ## Issue
      During the client input/output buffer processing, the memory usage is
      incrementally updated to keep track of clients going beyond a certain
      threshold `maxmemory-clients` to be evicted. However, this additional
      tracking activity leads to unnecessary CPU cycles wasted when no
      client-eviction is required. It is applicable in two cases.
      
      * `maxmemory-clients` is set to `0` which equates to no client eviction
        (applicable to all clients)
      * `CLIENT NO-EVICT` flag is set to `ON` which equates to a particular
        client not applicable for eviction.  
      
      ## Solution
      * Disable client memory usage tracking during the read/write flow when
        `maxmemory-clients` is set to `0` or `client no-evict` is `on`.
        The memory usage is tracked only during the `clientCron` i.e. it gets
        periodically updated.
      * Cleanup the clients from the memory usage bucket when client eviction
        is disabled.
      * When the maxmemory-clients config is enabled or disabled at runtime,
        we immediately update the memory usage buckets for all clients (tested
        scanning 80000 took some 20ms)
      
      Benchmark shown that this can improve performance by about 5% in
      certain situations.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      c0267b3f
  22. 06 Dec, 2022 2 commits
    • Viktor Söderqvist's avatar
      When converting a set to dict, presize for one more element to be added (#11559) · 8a315fc2
      Viktor Söderqvist authored
      
      
      In most cases when a listpack or intset is converted to a dict, the conversion
      is trigged when adding an element. The extra element is added after conversion
      to dict (in all cases except when the conversion is triggered by
      set-max-intset-entries being reached).
      
      If set-max-listpack-entries is set to a power of two, let's say 128, when
      adding the 129th element, the 128 element listpack is first converted to a dict
      with a hashtable presized for 128 elements. After converting to dict, the 129th
      element is added to the dict which immediately triggers incremental rehashing
      to size 256.
      
      This commit instead presizes the dict to one more element, with the assumption
      that conversion to dict is followed by adding another element, so the dict
      doesn't immediately need rehashing.
      Co-authored-by: default avatarsundb <sundbcn@gmail.com>
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      8a315fc2
    • Binbin's avatar
      Fix command line startup --sentinel problem (#11591) · 8f13ac10
      Binbin authored
      There is a issue with --sentinel:
      ```
      [root]# src/redis-server sentinel.conf --sentinel --loglevel verbose
      
      *** FATAL CONFIG FILE ERROR (Redis 255.255.255) ***
      Reading the configuration file, at line 352
      >>> 'sentinel "--loglevel" "verbose"'
      Unrecognized sentinel configuration statement
      ```
      
      This is because in #10660 (Redis 7.0.1), `--` prefix change break it.
      In this PR, we will handle `--sentinel` the same as we did for `--save`
      in #10866. i.e. it's a pseudo config option with no value.
      8f13ac10
  23. 05 Dec, 2022 1 commit
    • filipe oliveira's avatar
      GEOSEARCH BYBOX: Simplified haversine distance formula when longitude diff is 0 (#11579) · e48ac075
      filipe oliveira authored
      This is take 2 of `GEOSEARCH BYBOX` optimizations based on haversine
      distance formula when longitude diff is 0.
      The first one was in #11535 . 
      
      - Given longitude diff is 0 the asin(sqrt(a)) on the haversine is asin(sin(abs(u))).
      - arcsin(sin(x)) equal to x when x ∈[−𝜋/2,𝜋/2]. 
      - Given latitude is between [−𝜋/2,𝜋/2] we can simplifiy arcsin(sin(x)) to x.
      
      On the sample dataset with 60M datapoints, we've measured 55% increase
      in the achievable ops/sec.
      e48ac075