1. 09 Feb, 2017 1 commit
    • antirez's avatar
      Fix MIGRATE closing of cached socket on error. · f917e0da
      antirez authored
      After investigating issue #3796, it was discovered that MIGRATE
      could call migrateCloseSocket() after the original MIGRATE c->argv
      was already rewritten as a DEL operation. As a result the host/port
      passed to migrateCloseSocket() could be anything, often a NULL pointer
      that gets deferenced crashing the server.
      
      Now the socket is closed at an earlier time when there is a socket
      error in a later stage where no retry will be performed, before we
      rewrite the argument vector. Moreover a check was added so that later,
      in the socket_err label, there is no further attempt at closing the
      socket if the argument was rewritten.
      
      This fix should resolve the bug reported in #3796.
      f917e0da
  2. 14 Dec, 2016 1 commit
    • antirez's avatar
      MIGRATE: Remove upfront ttl initialization. · b53e73e1
      antirez authored
      After the fix for #3673 the ttl var is always initialized inside the
      loop itself, so the early initialization is not needed.
      
      Variables declaration also moved to a more local scope.
      b53e73e1
  3. 13 Dec, 2016 1 commit
    • antirez's avatar
      Replication: fix the infamous key leakage of writable slaves + EXPIRE. · 04542cff
      antirez authored
      BACKGROUND AND USE CASEj
      
      Redis slaves are normally write only, however the supprot a "writable"
      mode which is very handy when scaling reads on slaves, that actually
      need write operations in order to access data. For instance imagine
      having slaves replicating certain Sets keys from the master. When
      accessing the data on the slave, we want to peform intersections between
      such Sets values. However we don't want to intersect each time: to cache
      the intersection for some time often is a good idea.
      
      To do so, it is possible to setup a slave as a writable slave, and
      perform the intersection on the slave side, perhaps setting a TTL on the
      resulting key so that it will expire after some time.
      
      THE BUG
      
      Problem: in order to have a consistent replication, expiring of keys in
      Redis replication is up to the master, that synthesize DEL operations to
      send in the replication stream. However slaves logically expire keys
      by hiding them from read attempts from clients so that if the master did
      not promptly sent a DEL, the client still see logically expired keys
      as non existing.
      
      Because slaves don't actively expire keys by actually evicting them but
      just masking from the POV of read operations, if a key is created in a
      writable slave, and an expire is set, the key will be leaked forever:
      
      1. No DEL will be received from the master, which does not know about
      such a key at all.
      
      2. No eviction will be performed by the slave, since it needs to disable
      eviction because it's up to masters, otherwise consistency of data is
      lost.
      
      THE FIX
      
      In order to fix the problem, the slave should be able to tag keys that
      were created in the slave side and have an expire set in some way.
      
      My solution involved using an unique additional dictionary created by
      the writable slave only if needed. The dictionary is obviously keyed by
      the key name that we need to track: all the keys that are set with an
      expire directly by a client writing to the slave are tracked.
      
      The value in the dictionary is a bitmap of all the DBs where such a key
      name need to be tracked, so that we can use a single dictionary to track
      keys in all the DBs used by the slave (actually this limits the solution
      to the first 64 DBs, but the default with Redis is to use 16 DBs).
      
      This solution allows to pay both a small complexity and CPU penalty,
      which is zero when the feature is not used, actually. The slave-side
      eviction is encapsulated in code which is not coupled with the rest of
      the Redis core, if not for the hook to track the keys.
      
      TODO
      
      I'm doing the first smoke tests to see if the feature works as expected:
      so far so good. Unit tests should be added before merging into the
      4.0 branch.
      04542cff
  4. 08 Dec, 2016 1 commit
    • Jan-Erik Rediger's avatar
      Reset the ttl for additional keys · 2a32f037
      Jan-Erik Rediger authored
      Before, if a previous key had a TTL set but the current one didn't, the
      TTL was reused and thus resulted in wrong expirations set.
      
      This behaviour was experienced, when `MigrateDefaultPipeline` in
      redis-trib was set to >1
      
      Fixes #3655
      2a32f037
  5. 29 Nov, 2016 1 commit
  6. 16 Nov, 2016 1 commit
  7. 16 Jun, 2016 1 commit
    • antirez's avatar
      RESTORE: accept RDB dumps with older versions. · a3f893b8
      antirez authored
      Reference issue #3218.
      
      Checking the code I can't find a reason why the original RESTORE
      code was so opinionated about restoring only the current version. The
      code in to `rdb.c` appears to be capable as always to restore data from
      older versions of Redis, and the only places where it is needed the
      current version in order to correctly restore data, is while loading the
      opcodes, not the values itself as it happens in the case of RESTORE.
      
      For the above reasons, this commit enables RESTORE to accept older
      versions of values payloads.
      a3f893b8
  8. 05 May, 2016 1 commit
  9. 02 May, 2016 1 commit
    • antirez's avatar
      New masters with slots are now targets of migration if others are. · 4fdde78c
      antirez authored
      This fixes issue #3043.
      
      Before this fix, after a complete resharding of a master slots
      to other nodes, the master remains empty and the slaves migrate away
      to other masters with non-zero nodes. However the old master now empty,
      is no longer considered a target for migration, because the system has
      no way to tell it had slaves in the past.
      
      This fix leaves the algorithm used in the past untouched, but adds a
      new rule. When a new or old master which is empty and without slaves,
      are assigend with their first slot, if other masters in the cluster have
      slaves, they are automatically considered to be targets for replicas
      migration.
      4fdde78c
  10. 02 Feb, 2016 1 commit
  11. 01 Feb, 2016 1 commit
  12. 29 Jan, 2016 6 commits
  13. 28 Jan, 2016 1 commit
  14. 27 Jan, 2016 1 commit
  15. 26 Jan, 2016 3 commits
    • antirez's avatar
      fe44a7cb
    • antirez's avatar
    • antirez's avatar
      Better address udpate strategy when processing gossip sections. · 592419b4
      antirez authored
      The change covers the case where:
      
      1. There is a node we can't reach (in fail or pfail state).
      2. We see a different address for this node, in the gossip section sent
      to us by a node that, instead, is able to talk with the node we cannot
      talk to.
      
      In this case it's a good bet to switch to the address reported by this
      node, since there was an address switch and it is able to talk with the
      node and we are not.
      
      However previosuly this was done in a dangerous way, by initiating an
      handshake. The handshake, using the MEET packet, forces the receiver to
      join our cluster, and this is not a good idea. If the node in question
      really just switched address, but is the same node, it already knows about
      us, so we just need to perform an address update and a reconnection.
      
      So with this commit instead we just update the address of the node,
      release the node link if any, and attempt to reconnect in the next
      clusterCron() cycle.
      
      The commit also improves debugging messages printed by Cluster during
      address or ID switches.
      592419b4
  16. 19 Jan, 2016 2 commits
    • antirez's avatar
      Minor MIGRATE refactoring. · 83b862a3
      antirez authored
      Centralize cleanup of newargv in a single place.
      Add more comments to help a bit following a complex function.
      
      Related to issue #3016.
      83b862a3
    • antirez's avatar
      More variadic MIGRATE fixes. · f5a1e608
      antirez authored
      Another leak was fixed in the case of syntax error by restructuring the
      allocation strategy for the two dynamic vectors.
      
      We also make sure to always close the cached socket on I/O errors so that
      all the I/O errors are handled the same, even if we had a previously
      queued error of a different kind from the destination server.
      
      Thanks to Kevin McGehee. Related to issue #3016.
      f5a1e608
  17. 18 Jan, 2016 1 commit
    • antirez's avatar
      Various fixes to MIGRATE with multiple keys. · 00d3a40f
      antirez authored
      In issue #3016 Kevin McGehee identified multiple very serious issues in
      the new implementation of MIGRATE. This commit attempts to restructure
      the code in oder to avoid mistakes, an analysis of the new
      implementation is in progress in order to check for possible edge cases.
      00d3a40f
  18. 14 Jan, 2016 1 commit
    • antirez's avatar
      Cluster: fix setting nodes slaveof pointer to NULL on node release. · fc3ca8ff
      antirez authored
      With this commit we preserve the list of nodes that have .slaveof set
      to the node, even when the node is turned into a slave, and make sure to
      fix the .slaveof pointers to NULL when a node is freed from memory,
      regardless of the fact it's a slave or a master.
      
      Basically we try to remember the logical master in the current
      configuration even if the logical master advertised it as a slave
      already. However we still remember the associations, so that when a node
      is freed we can fix them.
      
      This should fix issue #3002.
      fc3ca8ff
  19. 11 Jan, 2016 3 commits
  20. 08 Jan, 2016 2 commits
  21. 06 Jan, 2016 1 commit
    • antirez's avatar
      Cluster: don't send -ASK to MIGRATE. · 00d637f2
      antirez authored
      For non existing keys, we don't want to send -ASK redirections to
      MIGRATE, since when moving slots from the migrating node to the
      importing node, we want just to ignore keys that are no longer there.
      They may be expired or deleted between the GETKEYSINSLOT call and the
      MIGRATE call. Otherwise this causes an error during migrations with
      redis-trib (or equivalent cluster management tools).
      00d637f2
  22. 16 Dec, 2015 1 commit
  23. 11 Dec, 2015 4 commits
    • antirez's avatar
      ac0a7310
    • antirez's avatar
      MIGRATE: fix replies processing and argument rewriting. · d85fc1e9
      antirez authored
      We need to process replies after errors in order to delete keys
      successfully transferred. Also argument rewriting was fixed since
      it was broken in several ways. Now a fresh argument vector is created
      and set if we are acknowledged of at least one key.
      d85fc1e9
    • antirez's avatar
      Pipelined multiple keys MIGRATE. · 9ebf7a67
      antirez authored
      9ebf7a67
    • antirez's avatar
      Cluster: replica migration with delay. · adc2fe69
      antirez authored
      We wait a fixed amount of time (5 seconds currently) much greater than
      the usual Cluster node to node communication latency, before migrating.
      This way when a failover occurs, before detecting the new master as a
      target for migration, we give the time to its natural slaves (the slaves
      of the failed over master) to announce they switched to the new master,
      preventing an useless migration operation.
      adc2fe69
  24. 10 Dec, 2015 1 commit
  25. 09 Dec, 2015 1 commit
    • antirez's avatar
      Fix replicas migration by adding a new flag. · e0f22df9
      antirez authored
      Some time ago I broken replicas migration (reported in #2924).
      The idea was to prevent masters without replicas from getting replicas
      because of replica migration, I remember it to create issues with tests,
      but there is no clue in the commit message about why it was so
      undesirable.
      
      However my patch as a side effect totally ruined the concept of replicas
      migration since we want it to work also for instances that, technically,
      never had slaves in the past: promoted slaves.
      
      So now instead the ability to be targeted by replicas migration, is a
      new flag "migrate-to". It only applies to masters, and is set in the
      following two cases:
      
      1. When a master gets a slave, it is set.
      2. When a slave turns into a master because of fail over, it is set.
      
      This way replicas migration targets are only masters that used to have
      slaves, and slaves of masters (that used to have slaves... obviously)
      and are promoted.
      
      The new flag is only internal, and is never exposed in the output nor
      persisted in the nodes configuration, since all the information to
      handle it are implicit in the cluster configuration we already have.
      e0f22df9
  26. 27 Nov, 2015 1 commit