1. 18 Nov, 2019 1 commit
  2. 15 Nov, 2019 2 commits
  3. 14 Nov, 2019 1 commit
  4. 22 Jul, 2019 1 commit
    • antirez's avatar
      Client side caching: call the invalidation functions always. · 842b44dc
      antirez authored
      Otherwise what happens is that the tracking table will never get garbage
      collected if there are no longer clients with tracking enabled.
      Now the invalidation function immediately checks if there is any table
      allocated, otherwise it returns ASAP, so the overhead when the feature
      is not used should be near zero.
      842b44dc
  5. 03 Jul, 2019 1 commit
  6. 21 Jun, 2018 1 commit
  7. 19 Feb, 2018 1 commit
    • antirez's avatar
      Track number of logically expired keys still in memory. · ffde73c5
      antirez authored
      This commit adds two new fields in the INFO output, stats section:
      
      expired_stale_perc:0.34
      expired_time_cap_reached_count:58
      
      The first field is an estimate of the number of keys that are yet in
      memory but are already logically expired. They reason why those keys are
      yet not reclaimed is because the active expire cycle can't spend more
      time on the process of reclaiming the keys, and at the same time nobody
      is accessing such keys. However as the active expire cycle runs, while
      it will eventually have to return to the caller, because of time limit
      or because there are less than 25% of keys logically expired in each
      given database, it collects the stats in order to populate this INFO
      field.
      
      Note that expired_stale_perc is a running average, where the current
      sample accounts for 5% and the history for 95%, so you'll see it
      changing smoothly over time.
      
      The other field, expired_time_cap_reached_count, counts the number
      of times the expire cycle had to stop, even if still it was finding a
      sizeable number of keys yet to expire, because of the time limit.
      This allows people handling operations to understand if the Redis
      server, during mass-expiration events, is able to collect keys fast
      enough usually. It is normal for this field to increment during mass
      expires, but normally it should very rarely increment. When instead it
      constantly increments, it means that the current workloads is using
      a very important percentage of CPU time to expire keys.
      
      This feature was created thanks to the hints of Rashmi Ramesh and
      Bart Robinson from Twitter. In private email exchanges, they noted how
      it was important to improve the observability of this parameter in the
      Redis server. Actually in big deployments, the amount of keys that are
      yet to expire in each server, even if they are logically expired, may
      account for a very big amount of wasted memory.
      ffde73c5
  8. 21 Nov, 2017 1 commit
  9. 23 Jun, 2017 1 commit
    • antirez's avatar
      Issue #4027: unify comment and modify return value in freeMemoryIfNeeded(). · c9097393
      antirez authored
      It looks safer to return C_OK from freeMemoryIfNeeded() when clients are
      paused because returning C_ERR may prevent success of writes. It is
      possible that there is no difference in practice since clients cannot
      execute writes while clients are paused, but it looks more correct this
      way, at least conceptually.
      
      Related to PR #4028.
      c9097393
  10. 13 Jun, 2017 1 commit
  11. 01 Jun, 2017 1 commit
  12. 08 Apr, 2017 1 commit
  13. 13 Dec, 2016 4 commits
    • antirez's avatar
      Writable slaves expires: fix leak in key tracking. · b6f871cf
      antirez authored
      We need to use a dictionary type that frees the key, since we copy the
      keys in the dictionary we use to track expires created in the slave
      side.
      b6f871cf
    • antirez's avatar
      INFO: show num of slave-expires keys tracked. · d1adc85a
      antirez authored
      d1adc85a
    • antirez's avatar
      Fix created->created typo in expire.c · 5b9ba264
      antirez authored
      5b9ba264
    • antirez's avatar
      Replication: fix the infamous key leakage of writable slaves + EXPIRE. · 04542cff
      antirez authored
      BACKGROUND AND USE CASEj
      
      Redis slaves are normally write only, however the supprot a "writable"
      mode which is very handy when scaling reads on slaves, that actually
      need write operations in order to access data. For instance imagine
      having slaves replicating certain Sets keys from the master. When
      accessing the data on the slave, we want to peform intersections between
      such Sets values. However we don't want to intersect each time: to cache
      the intersection for some time often is a good idea.
      
      To do so, it is possible to setup a slave as a writable slave, and
      perform the intersection on the slave side, perhaps setting a TTL on the
      resulting key so that it will expire after some time.
      
      THE BUG
      
      Problem: in order to have a consistent replication, expiring of keys in
      Redis replication is up to the master, that synthesize DEL operations to
      send in the replication stream. However slaves logically expire keys
      by hiding them from read attempts from clients so that if the master did
      not promptly sent a DEL, the client still see logically expired keys
      as non existing.
      
      Because slaves don't actively expire keys by actually evicting them but
      just masking from the POV of read operations, if a key is created in a
      writable slave, and an expire is set, the key will be leaked forever:
      
      1. No DEL will be received from the master, which does not know about
      such a key at all.
      
      2. No eviction will be performed by the slave, since it needs to disable
      eviction because it's up to masters, otherwise consistency of data is
      lost.
      
      THE FIX
      
      In order to fix the problem, the slave should be able to tag keys that
      were created in the slave side and have an expire set in some way.
      
      My solution involved using an unique additional dictionary created by
      the writable slave only if needed. The dictionary is obviously keyed by
      the key name that we need to track: all the keys that are set with an
      expire directly by a client writing to the slave are tracked.
      
      The value in the dictionary is a bitmap of all the DBs where such a key
      name need to be tracked, so that we can use a single dictionary to track
      keys in all the DBs used by the slave (actually this limits the solution
      to the first 64 DBs, but the default with Redis is to use 16 DBs).
      
      This solution allows to pay both a small complexity and CPU penalty,
      which is zero when the feature is not used, actually. The slave-side
      eviction is encapsulated in code which is not coupled with the rest of
      the Redis core, if not for the hook to track the keys.
      
      TODO
      
      I'm doing the first smoke tests to see if the feature works as expected:
      so far so good. Unit tests should be added before merging into the
      4.0 branch.
      04542cff
  14. 06 Jul, 2016 1 commit