1. 09 May, 2017 1 commit
  2. 04 May, 2017 2 commits
  3. 03 May, 2017 8 commits
  4. 02 May, 2017 3 commits
  5. 28 Apr, 2017 2 commits
  6. 27 Apr, 2017 1 commit
    • antirez's avatar
      PSYNC2: fix master cleanup when caching it. · 469d6e2b
      antirez authored
      The master client cleanup was incomplete: resetClient() was missing and
      the output buffer of the client was not reset, so pending commands
      related to the previous connection could be still sent.
      
      The first problem caused the client argument vector to be, at times,
      half populated, so that when the correct replication stream arrived the
      protcol got mixed to the arugments creating invalid commands that nobody
      called.
      
      Thanks to @yangsiran for also investigating this problem, after
      already providing important design / implementation hints for the
      original PSYNC2 issues (see referenced Github issue).
      
      Note that this commit adds a new function to the list library of Redis
      in order to be able to reset a list without destroying it.
      
      Related to issue #3899.
      469d6e2b
  7. 22 Apr, 2017 4 commits
  8. 21 Apr, 2017 1 commit
    • antirez's avatar
      Check event loop creation return value. Fix #3951. · 238cebdd
      antirez authored
      Normally we never check for OOM conditions inside Redis since the
      allocator will always return a pointer or abort the program on OOM
      conditons. However we cannot have control on epool_create(), that may
      fail for kernel OOM (according to the manual page) even if all the
      parameters are correct, so the function aeCreateEventLoop() may indeed
      return NULL and this condition must be checked.
      238cebdd
  9. 20 Apr, 2017 1 commit
  10. 19 Apr, 2017 3 commits
    • antirez's avatar
      Fix getKeysUsingCommandTable() in cluster mode. · 7d9dd80d
      antirez authored
      Close #3940.
      7d9dd80d
    • antirez's avatar
      PSYNC2: discard pending transactions from cached master. · 189a12af
      antirez authored
      During the review of the fix for #3899, @yangsiran identified an
      implementation bug: given that the offset is now relative to the applied
      part of the replication log, when we cache a master, the successive
      PSYNC2 request will be made in order to *include* the transaction that
      was not completely processed. This means that we need to discard any
      pending transaction from our replication buffer: it will be re-executed.
      189a12af
    • antirez's avatar
      Fix PSYNC2 incomplete command bug as described in #3899. · 22be435e
      antirez authored
      This bug was discovered by @kevinmcgehee and constituted a major hidden
      bug in the PSYNC2 implementation, caused by the propagation from the
      master of incomplete commands to slaves.
      
      The bug had several results:
      
      1. Borrowing from Kevin text in the issue: "Given that slaves blindly
      copy over their master's input into their own replication backlog over
      successive read syscalls, it's possible that with large commands or
      small TCP buffers, partial commands are present in this buffer. If the
      master were to fail before successfully propagating the entire command
      to a slave, the slaves will never execute the partial command (since the
      client is invalidated) but will copy it to replication backlog which may
      relay those invalid bytes to its slaves on PSYNC2, corrupting the
      backlog and possibly other valid commands that follow the failover.
      Simple command boundaries aren't sufficient to capture this, either,
      because in the case of a MULTI/EXEC block, if the master successfully
      propagates a subset of the commands but not the EXEC, then the
      transaction in the backlog becomes corrupt and could corrupt other
      slaves that consume this data."
      
      2. As identified by @yangsiran later, there is another effect of the
      bug. For the same mechanism of the first problem, a slave having another
      slave, could receive a full resynchronization request with an already
      half-applied command in the backlog. Once the RDB is ready, it will be
      sent to the slave, and the replication will continue sending to the
      sub-slave the other half of the command, which is not valid.
      
      The fix, designed by @yangsiran and @antirez, and implemented by
      @antirez, uses a secondary buffer in order to feed the sub-masters and
      update the replication backlog and offsets, only when a given part of
      the query buffer is actually *applied* to the state of the instance,
      that is, when the command gets processed and the command is not pending
      in the Redis transaction buffer because of CLIENT_MULTI state.
      
      Given that now the backlog and offsets representation are in agreement
      with the actual processed commands, both issue 1 and 2 should no longer
      be possible.
      
      Thanks to @kevinmcgehee, @yangsiran and @oranagra for their work in
      identifying and designing a fix for this problem.
      22be435e
  11. 18 Apr, 2017 8 commits
  12. 17 Apr, 2017 2 commits
  13. 15 Apr, 2017 1 commit
    • antirez's avatar
      Cluster: discard pong times in the future. · 271733f4
      antirez authored
      However we allow for 500 milliseconds of tolerance, in order to
      avoid often discarding semantically valid info (the node is up)
      because of natural few milliseconds desync among servers even when
      NTP is used.
      
      Note that anyway we should ping the node from time to time regardless and
      discover if it's actually down from our point of view, since no update
      is accepted while we have an active ping on the node.
      
      Related to #3929.
      271733f4
  14. 14 Apr, 2017 3 commits
    • antirez's avatar
      Test: fix, hopefully, false PSYNC failure like in issue #2715. · 3f068b92
      antirez authored
      And many other related Github issues... all reporting the same problem.
      There was probably just not enough backlog in certain unlucky runs.
      I'll ask people that can reporduce if they see now this as fixed as
      well.
      3f068b92
    • antirez's avatar
      Cluster: always add PFAIL nodes at end of gossip section. · 02777bb2
      antirez authored
      To rely on the fact that nodes in PFAIL state will be shared around by
      randomly adding them in the gossip section is a weak assumption,
      especially after changes related to sending less ping/pong packets.
      
      We want to always include gossip entries for all the nodes that are in
      PFAIL state, so that the PFAIL -> FAIL state promotion can happen much
      faster and reliably.
      
      Related to #3929.
      02777bb2
    • antirez's avatar
      Cluster: fix gossip section ping/pong times encoding. · 8c829d9e
      antirez authored
      The gossip section times are 32 bit, so cannot store the milliseconds
      time but just the seconds approximation, which is good enough for our
      uses. At the same time however, when comparing the gossip section times
      of other nodes with our node's view, we need to convert back to
      milliseconds.
      
      Related to #3929. Without this change the patch to reduce the traffic in
      the bus message does not work.
      8c829d9e