1. 16 Dec, 2016 7 commits
  2. 14 Dec, 2016 3 commits
  3. 13 Dec, 2016 5 commits
    • antirez's avatar
      Writable slaves expires: unit test. · dee11eba
      antirez authored
      dee11eba
    • antirez's avatar
      Writable slaves expires: fix leak in key tracking. · b6f871cf
      antirez authored
      We need to use a dictionary type that frees the key, since we copy the
      keys in the dictionary we use to track expires created in the slave
      side.
      b6f871cf
    • antirez's avatar
      INFO: show num of slave-expires keys tracked. · d1adc85a
      antirez authored
      d1adc85a
    • antirez's avatar
      Fix created->created typo in expire.c · 5b9ba264
      antirez authored
      5b9ba264
    • antirez's avatar
      Replication: fix the infamous key leakage of writable slaves + EXPIRE. · 04542cff
      antirez authored
      BACKGROUND AND USE CASEj
      
      Redis slaves are normally write only, however the supprot a "writable"
      mode which is very handy when scaling reads on slaves, that actually
      need write operations in order to access data. For instance imagine
      having slaves replicating certain Sets keys from the master. When
      accessing the data on the slave, we want to peform intersections between
      such Sets values. However we don't want to intersect each time: to cache
      the intersection for some time often is a good idea.
      
      To do so, it is possible to setup a slave as a writable slave, and
      perform the intersection on the slave side, perhaps setting a TTL on the
      resulting key so that it will expire after some time.
      
      THE BUG
      
      Problem: in order to have a consistent replication, expiring of keys in
      Redis replication is up to the master, that synthesize DEL operations to
      send in the replication stream. However slaves logically expire keys
      by hiding them from read attempts from clients so that if the master did
      not promptly sent a DEL, the client still see logically expired keys
      as non existing.
      
      Because slaves don't actively expire keys by actually evicting them but
      just masking from the POV of read operations, if a key is created in a
      writable slave, and an expire is set, the key will be leaked forever:
      
      1. No DEL will be received from the master, which does not know about
      such a key at all.
      
      2. No eviction will be performed by the slave, since it needs to disable
      eviction because it's up to masters, otherwise consistency of data is
      lost.
      
      THE FIX
      
      In order to fix the problem, the slave should be able to tag keys that
      were created in the slave side and have an expire set in some way.
      
      My solution involved using an unique additional dictionary created by
      the writable slave only if needed. The dictionary is obviously keyed by
      the key name that we need to track: all the keys that are set with an
      expire directly by a client writing to the slave are tracked.
      
      The value in the dictionary is a bitmap of all the DBs where such a key
      name need to be tracked, so that we can use a single dictionary to track
      keys in all the DBs used by the slave (actually this limits the solution
      to the first 64 DBs, but the default with Redis is to use 16 DBs).
      
      This solution allows to pay both a small complexity and CPU penalty,
      which is zero when the feature is not used, actually. The slave-side
      eviction is encapsulated in code which is not coupled with the rest of
      the Redis core, if not for the hook to track the keys.
      
      TODO
      
      I'm doing the first smoke tests to see if the feature works as expected:
      so far so good. Unit tests should be added before merging into the
      4.0 branch.
      04542cff
  4. 12 Dec, 2016 2 commits
  5. 08 Dec, 2016 1 commit
    • Jan-Erik Rediger's avatar
      Reset the ttl for additional keys · 2a32f037
      Jan-Erik Rediger authored
      Before, if a previous key had a TTL set but the current one didn't, the
      TTL was reused and thus resulted in wrong expirations set.
      
      This behaviour was experienced, when `MigrateDefaultPipeline` in
      redis-trib was set to >1
      
      Fixes #3655
      2a32f037
  6. 06 Dec, 2016 2 commits
  7. 05 Dec, 2016 5 commits
    • antirez's avatar
      Modules: types doc updated to new API. · 16cce320
      antirez authored
      16cce320
    • antirez's avatar
      Modules: API doc updated (auto generated). · 37b6e16a
      antirez authored
      37b6e16a
    • antirez's avatar
    • antirez's avatar
      Geo: improve fuzz test. · b1fc06f7
      antirez authored
      The test now uses more diverse radius sizes, especially sizes near or
      greater the whole earth surface are used, that are known to trigger edge
      cases. Moreover the PRNG seeding was probably resulting into the same
      sequence tested over and over again, now seeding unsing the current unix
      time in milliseconds.
      
      Related to #3631.
      b1fc06f7
    • antirez's avatar
      Geo: fix computation of bounding box. · 001138ae
      antirez authored
      A bug was reported in the context in issue #3631. The root cause of the
      bug was that certain neighbor boxes were zeroed after the "inside the
      bounding box or not" check, simply because the bounding box computation
      function was wrong.
      
      A few debugging infos where enhanced and moved in other parts of the
      code. A check to avoid steps=0 was added, but is unrelated to this
      issue and I did not verified it was an actual bug in practice.
      001138ae
  8. 02 Dec, 2016 1 commit
  9. 01 Dec, 2016 2 commits
  10. 30 Nov, 2016 4 commits
  11. 29 Nov, 2016 5 commits
  12. 28 Nov, 2016 2 commits
    • antirez's avatar
      PSYNC2: stop sending newlines to sub-slaves when master is down. · eab865a0
      antirez authored
      This actually includes two changes:
      
      1) No newlines to take the master-slave link up when the upstream master
      is down. Doing this is dangerous because the sub-slave often is received
      replication protocol for an half-command, so can't receive newlines
      without desyncing the replication link, even with the code in order to
      cancel out the bytes that PSYNC2 was using. Moreover this is probably
      also not needed/sane, because anyway the slave can keep serving
      requests, and because if it's configured to don't serve stale data, it's
      a good idea, actually, to break the link.
      
      2) When a +CONTINUE with a different ID is received, we now break
      connection with the sub-slaves: they need to be notified as well. This
      was part of the original specification but for some reason it was not
      implemented in the code, and was alter found as a PSYNC2 bug in the
      integration testing.
      eab865a0
    • antirez's avatar
      PSYNC2: Test (WIP). · 16559a02
      antirez authored
      This is the PSYNC2 test that helped find issues in the code, and that
      still can show a protocol desync from time to time. Work is in progress
      in order to find the issue. For now the test is not enabled in "make
      test" and must be run manually.
      16559a02
  13. 25 Nov, 2016 1 commit