• guybe7's avatar
    X[AUTO]CLAIM should skip deleted entries (#10227) · 3c3e6cc1
    guybe7 authored
    Fix #7021 #8924 #10198
    
    # Intro
    Before this commit X[AUTO]CLAIM used to transfer deleted entries from one
    PEL to another, but reply with "nil" for every such entry (instead of the entry id).
    The idea (for XCLAIM) was that the caller could see this "nil", realize the entry
    no longer exists, and XACK it in order to remove it from PEL.
    The main problem with that approach is that it assumes there's a correlation
    between the index of the "id" arguments and the array indices, which there
    isn't (in case some of the input IDs to XCLAIM never existed/read):
    
    ```
    127.0.0.1:6379> XADD x 1 f1 v1
    "1-0"
    127.0.0.1:6379> XADD x 2 f1 v1
    "2-0"
    127.0.0.1:6379> XADD x 3 f1 v1
    "3-0"
    127.0.0.1:6379> XGROUP CREATE x grp 0
    OK
    127.0.0.1:6379> XREADGROUP GROUP grp Alice COUNT 2 STREAMS x >
    1) 1) "x"
       2) 1) 1) "1-0"
             2) 1) "f1"
                2) "v1"
          2) 1) "2-0"
             2) 1) "f1"
                2) "v1"
    127.0.0.1:6379> XDEL x 1 2
    (integer) 2
    127.0.0.1:6379> XCLAIM x grp Bob 0 0-99 1-0 1-99 2-0
    1) (nil)
    2) (nil)
    ```
    
    # Changes
    Now,  X[AUTO]CLAIM acts in the following way:
    1. If one tries to claim a deleted entry, we delete it from the PEL we found it in
      (and the group PEL too). So de facto, such entry is not claimed, just cleared
      from PEL (since anyway it doesn't exist in the stream)
    2. since we never claim deleted entries, X[AUTO]CLAIM will never return "nil"
      instead of an entry.
    3. add a new element to XAUTOCLAIM's response (see below)
    
    # Knowing which entries were cleared from the PEL
    The caller may want to log any entries that were found in a PEL but deleted from
    the stream itself (it would suggest that there might be a bug in the application:
    trimming the stream while some entries were still no processed by the consumers)
    
    ## XCLAIM
    the set {XCLAIM input ids} - {XCLAIM returned ids} contains all the entry ids that were
    not claimed which means they were deleted (assuming the input contains only entries
    from some PEL). The user doesn't need to XACK them because XCLAIM had already
    deleted them from the source PEL.
    
    ## XAUTOCLAIM
    XAUTOCLAIM has a new element added to its reply: it's an array of all the deleted
    stream IDs it stumbled upon.
    
    This is somewhat of a breaking change since X[AUTO]CLAIM used to be able to reply
    with "nil" and now it can't... But since it was undocumented (and generally a bad idea
    to rely on it, as explained above) the breakage is not that bad.
    3c3e6cc1
xautoclaim.json 2.1 KB