1. 22 Jun, 2023 1 commit
    • guybe7's avatar
      Modules: Unblock from within a timer coverage (#12337) · 32301999
      guybe7 authored
      Apart from adding the missing coverage, this PR also adds `blockedBeforeSleep`
      that gathers all block-related functions from `beforeSleep`
      
      The order inside `blockedBeforeSleep` is different: now `handleClientsBlockedOnKeys`
      (which may unblock clients) is called before `processUnblockedClients` (which handles
      unblocked clients).
      It makes sense to have this order.
      
      There are no visible effects of the wrong ordering, except some cleanups of the now-unblocked
      client would have  happen in the next `beforeSleep` (will now happen in the current one)
      
      The reason we even got into it is because i triggers an assertion in logresreq.c (breaking
      the assumption that `unblockClient` is called **before** actually flushing the reply to the socket):
      `handleClientsBlockedOnKeys` is called, then it calls `moduleUnblockClientOnKey`, which calls
      `moduleUnblockClient`, which adds the client to `moduleUnblockedClients` back to `beforeSleep`,
      we call `handleClientsWithPendingWritesUsingThreads`, it writes the data of buf to the client, so
      `client->bufpos` became 0
      On the next `beforeSleep`, we call `moduleHandleBlockedClients`, which calls `unblockClient`,
      which calls `reqresAppendResponse`, triggering the assert. (because the `bufpos` is 0) - see https://github.com/redis/redis/pull/12301#discussion_r1226386716
      32301999
  2. 28 May, 2023 1 commit
    • Oran Agra's avatar
      Fix WAIT for clients being blocked in a module command (#12220) · 6117f288
      Oran Agra authored
      So far clients being blocked and unblocked by a module command would
      update the c->woff variable and so WAIT was ineffective and got released
      without waiting for the command actions to propagate.
      
      This seems to have existed since forever, but not for RM_BlockClientOnKeys.
      
      It is problematic though to know if the module did or didn't propagate
      anything in that command, so for now, instead of adding an API, we'll
      just update the woff to the latest offset when unblocking, this will
      cause the client to possibly wait excessively, but that's not that bad.
      6117f288
  3. 08 Mar, 2023 1 commit
    • ranshid's avatar
      Fix an issue when module decides to unblock a client which is blocked on keys (#11832) · 4988b928
      ranshid authored
      Currently (starting at #11012) When a module is blocked on keys it sets the
      CLIENT_PENDING_COMMAND flag.
      However in case the module decides to unblock the client not via the regular flow
      (eg timeout, key signal or CLIENT UNBLOCK command) it will attempt to reprocess the
      module command and potentially blocked again.
      
      This fix remove the CLIENT_PENDING_COMMAND flag in case blockedForKeys is
      issued from module context.
      4988b928
  4. 14 Feb, 2023 1 commit
    • guybe7's avatar
      Minor changes around the blockonkeys test module (#11803) · 9483ab0b
      guybe7 authored
      All of the POP commands must not decr length below 0.
      So, get_fsl will delete the key if the length is 0 (unless
      the caller wished to create if doesn't exist)
      
      Other:
      1. Use REDISMODULE_WRITE where needed (POP commands)
      2. Use wait_for_blokced_clients in test
      
      Unrelated:
      Use quotes instead of curly braces in zset.tcl, for variable expansion
      9483ab0b
  5. 18 Oct, 2022 1 commit
    • guybe7's avatar
      Blocked module clients should be aware when a key is deleted (#11310) · b57fd010
      guybe7 authored
      The use case is a module that wants to implement a blocking command on a key that
      necessarily exists and wants to unblock the client in case the key is deleted (much like
      what we implemented for XREADGROUP in #10306)
      
      New module API:
      * RedisModule_BlockClientOnKeysWithFlags
      
      Flags:
      * REDISMODULE_BLOCK_UNBLOCK_NONE
      * REDISMODULE_BLOCK_UNBLOCK_DELETED
      
      ### Detailed description of code changes
      
      blocked.c:
      1. Both module and stream functions are called whether the key exists or not, regardless of
        its type. We do that in order to allow modules/stream to unblock the client in case the key
        is no longer present or has changed type (the behavior for streams didn't change, just code
        that moved into serveClientsBlockedOnStreamKey)
      2. Make sure afterCommand is called in serveClientsBlockedOnKeyByModule, in order to propagate
        actions from moduleTryServeClientBlockedOnKey.
      3. handleClientsBlockedOnKeys: call propagatePendingCommands directly after lookupKeyReadWithFlags
        to prevent a possible lazy-expire DEL from being mixed with any command propagated by the
        preceding functions.
      4. blockForKeys: Caller can specifiy that it wants to be awakened if key is deleted.
         Minor optimizations (use dictAddRaw).
      5. signalKeyAsReady became signalKeyAsReadyLogic which can take a boolean in case the key is deleted.
        It will only signal if there's at least one client that awaits key deletion (to save calls to
        handleClientsBlockedOnKeys).
        Minor optimizations (use dictAddRaw)
      
      db.c:
      1. scanDatabaseForDeletedStreams is now scanDatabaseForDeletedKeys and will signalKeyAsReady
        for any key that was removed from the database or changed type. It is the responsibility of the code
        in blocked.c to ignore or act on deleted/type-changed keys.
      2. Use the new signalDeletedKeyAsReady where needed
      
      blockedonkey.c + tcl:
      1. Added test of new capabilities (FSL.BPOPGT now requires the key to exist in order to work)
      b57fd010
  6. 08 Mar, 2022 1 commit
    • guybe7's avatar
      XREADGROUP: Unblock client if stream is deleted (#10306) · 2a295408
      guybe7 authored
      Deleting a stream while a client is blocked XREADGROUP should unblock the client.
      
      The idea is that if a client is blocked via XREADGROUP is different from
      any other blocking type in the sense that it depends on the existence of both
      the key and the group. Even if the key is deleted and then revived with XADD
      it won't help any clients blocked on XREADGROUP because the group no longer
      exist, so they would fail with -NOGROUP anyway.
      The conclusion is that it's better to unblock these clients (with error) upon
      the deletion of the key, rather than waiting for the first XADD. 
      
      Other changes:
      1. Slightly optimize all `serveClientsBlockedOn*` functions by checking `server.blocked_clients_by_type`
      2. All `serveClientsBlockedOn*` functions now use a list iterator rather than looking at `listFirst`, relying
        on `unblockClient` to delete the head of the list. Before this commit, only `serveClientsBlockedOnStreams`
        used to work like that.
      3. bugfix: CLIENT UNBLOCK ERROR should work even if the command doesn't have a timeout_callback
        (only relevant to module commands)
      2a295408
  7. 22 Jan, 2021 1 commit
    • Viktor Söderqvist's avatar
      Test that module can wake up module blocked on non-empty list key (#8382) · 9c148310
      Viktor Söderqvist authored
      BLPOP and other blocking list commands can only block on empty keys
      and LPUSH only wakes up clients when the list is created.
      
      Using the module API, it's possible to block on a non-empty key.
      Unblocking a client blocked on a non-empty list (or zset) can only
      be done using RedisModule_SignalKeyAsReady(). This commit tests it.
      9c148310
  8. 20 Jan, 2021 1 commit
    • guybe7's avatar
      Fix misleading module test (#8366) · 5a77d015
      guybe7 authored
      the test was misleading because the module would actually woke up on a wrong type and
      re-blocked, while the test name suggests the module doesn't not wake up at all on a wrong type..
      
      i changed the name of the test + added verification that indeed the module wakes up and gets
      re-blocked after it understand it's the wrong type
      5a77d015
  9. 19 Jan, 2021 1 commit
    • Viktor Söderqvist's avatar
      Bugfix: Make modules blocked on keys unblock on commands like LPUSH (#8356) · 4985c11b
      Viktor Söderqvist authored
      This was a regression from #7625 (only in 6.2 RC2).
      
      This makes it possible again to implement blocking list and zset
      commands using the modules API.
      
      This commit also includes a test case for the reverse: A module
      unblocks a client blocked on BLPOP by inserting elements using
      RedisModule_ListPush(). This already works, but it was untested.
      4985c11b
  10. 03 Apr, 2020 1 commit
    • Guy Benoish's avatar
      Try to fix time-sensitive tests in blockonkey.tcl · 1b0d30ae
      Guy Benoish authored
      There is an inherent race between the deferring client and the
      "main" client of the test: While the deferring client issues a blocking
      command, we can't know for sure that by the time the "main" client
      tries to issue another command (Usually one that unblocks the deferring
      client) the deferring client is even blocked...
      For lack of a better choice this commit uses TCL's 'after' in order
      to give some time for the deferring client to issues its blocking
      command before the "main" client does its thing.
      This problem probably exists in many other tests but this commit
      tries to fix blockonkeys.tcl
      1b0d30ae
  11. 01 Apr, 2020 1 commit
    • Guy Benoish's avatar
      Fix memory corruption in moduleHandleBlockedClients · c4dc5b80
      Guy Benoish authored
      By using a "circular BRPOPLPUSH"-like scenario it was
      possible the get the same client on db->blocking_keys
      twice (See comment in moduleTryServeClientBlockedOnKey)
      
      The fix was actually already implememnted in
      moduleTryServeClientBlockedOnKey but it had a bug:
      the funxction should return 0 or 1 (not OK or ERR)
      
      Other changes:
      1. Added two commands to blockonkeys.c test module (To
         reproduce the case described above)
      2. Simplify blockonkeys.c in order to make testing easier
      3. cast raxSize() to avoid warning with format spec
      c4dc5b80
  12. 30 Dec, 2019 1 commit
    • Guy Benoish's avatar
      Modules: Fix blocked-client-related memory leak · d7d13721
      Guy Benoish authored
      If a blocked module client times-out (or disconnects, unblocked
      by CLIENT command, etc.) we need to call moduleUnblockClient
      in order to free memory allocated by the module sub-system
      and blocked-client private data
      
      Other changes:
      Made blockedonkeys.tcl tests a bit more aggressive in order
      to smoke-out potential memory leaks
      d7d13721
  13. 04 Nov, 2019 1 commit