1. 26 Jul, 2015 3 commits
  2. 12 Jun, 2015 1 commit
    • antirez's avatar
      Sentinel: fix bug in config rewriting during failover · 821a9866
      antirez authored
      We have a check to rewrite the config properly when a failover is in
      progress, in order to add the current (already failed over) master as
      slave, and don't include in the slave list the promoted slave itself.
      
      However there was an issue, the variable with the right address was
      computed but never used when the code was modified, and no tests are
      available for this feature for two reasons:
      
      1. The Sentinel unit test currently does not test Sentinel ability to
      persist its state at all.
      2. It is a very hard to trigger state since it lasts for little time in
      the context of the testing framework.
      
      However this feature should be covered in the test in some way.
      
      The bug was found by @badboy using the clang static analyzer.
      
      Effects of the bug on safety of Sentinel
      ===
      
      This bug results in severe issues in the following case:
      
      1. A Sentinel is elected leader.
      2. During the failover, it persists a wrong config with a known-slave
      entry listing the master address.
      3. The Sentinel crashes and restarts, reading invalid configuration from
      disk.
      4. It sees that the slave now does not obey the logical configuration
      (should replicate from the current master), so it sends a SLAVEOF
      command to the master (since the slave master is the same) creating a
      replication loop (attempt to replicate from itself) which Redis is
      currently unable to detect.
      5. This means that the master is no longer available because of the bug.
      
      However the lack of availability should be only transient (at least
      in my tests, but other states could be possible where the problem
      is not recovered automatically) because:
      
      6. Sentinels treat masters reporting to be slaves as failing.
      7. A new failover is triggered, and a slave is promoted to master.
      
      Bug lifetime
      ===
      
      The bug is there forever. Commit 16237d78 actually tried to fix the bug
      but in the wrong way (the computed variable was never used! My fault).
      So this bug is there basically since the start of Sentinel.
      
      Since the bug is hard to trigger, I remember little reports matching
      this condition, but I remember at least a few. Also in automated tests
      where instances were stopped and restarted multiple times automatically
      I remember hitting this issue, however I was not able to reproduce nor
      to determine with the information I had at the time what was causing the
      issue.
      821a9866
  3. 25 May, 2015 2 commits
  4. 22 May, 2015 1 commit
  5. 20 May, 2015 1 commit
  6. 18 May, 2015 1 commit
    • antirez's avatar
      Sentinel: SENTINEL CKQUORUM command · abc65e89
      antirez authored
      A way for monitoring systems to check that Sentinel is technically able
      to reach the quorum and failover, using the currently visible Sentinels.
      abc65e89
  7. 15 May, 2015 1 commit
  8. 14 May, 2015 7 commits
  9. 13 May, 2015 1 commit
  10. 12 May, 2015 3 commits
  11. 11 May, 2015 3 commits
  12. 08 May, 2015 4 commits
  13. 07 May, 2015 1 commit
    • antirez's avatar
      Sentinel: don't detect duplicated Sentinels, just address switch · a0cd75cd
      antirez authored
      Since with a previous commit Sentinels now persist their unique ID, we
      no longer need to detect duplicated Sentinels and re-add them. We remove
      and re-add back using different events only in the case of address
      switch of the same Sentinel, without generating a new +sentinel event.
      a0cd75cd
  14. 06 May, 2015 1 commit
    • antirez's avatar
      Sentinel: persist its unique ID across restarts. · 794fc4c9
      antirez authored
      Previously Sentinels always changed unique ID across restarts, relying
      on the server.runid field. This is not a good idea, and forced Sentinel
      to rely on detection of duplicated Sentinels and a potentially dangerous
      clean-up and re-add operation of the Sentinel instance that was
      rebooted.
      
      Now the ID is generated at the first start and persisted in the
      configuration file, so that a given Sentinel will have its unique
      ID forever (unless the configuration is manually deleted or there is a
      filesystem corruption).
      794fc4c9
  15. 04 May, 2015 2 commits
    • therealbill's avatar
      Making sentinel flush config on +slave · cc799d25
      therealbill authored
      Originally, only the +slave event which occurs when a slave is
      reconfigured during sentinelResetMasterAndChangeAddress triggers a flush
      of the config to disk.  However, newly discovered slaves don't
      apparently trigger this flush but do trigger the +slave event issuance.
      
      So if you start up a sentinel, add a master, then add a slave to the
      master (as a way to reproduce it) you'll see the +slave event issued,
      but the sentinel config won't be updated with the known-slave entry.
      
      This change makes sentinel do the flush of the config if a new slave is
      deteted in sentinelRefreshInstanceInfo.
      cc799d25
    • antirez's avatar
      Sentinel: remove useless sentinelFlushConfig() call · 99c93f34
      antirez authored
      To rewrite the config in the loop that adds slaves back after a master
      reset, in order to handle switching to another master, is useless: it
      just adds latency since there is an fsync call in the inner loop,
      without providing any additional guarantee, but the contrary, since if
      after the first loop iteration the server crashes we end with just a
      single slave entry losing all the other informations.
      
      It is wiser to rewrite the config at the end when the full new
      state is configured.
      99c93f34
  16. 28 Apr, 2015 1 commit
  17. 02 Feb, 2015 1 commit
    • Leandro López (inkel)'s avatar
      Support CLIENT commands in Redis Sentinel · d5e01519
      Leandro López (inkel) authored
      When trying to debug sentinel connections or max connections errors it
      would be very useful to have the ability to see the list of connected
      clients to a running sentinel. At the same time it would be very helpful
      to be able to name each sentinel connection or kill offending clients.
      
      This commits adds the already defined CLIENT commands back to Redis
      Sentinel.
      d5e01519
  18. 23 Dec, 2014 3 commits
  19. 11 Dec, 2014 3 commits
    • antirez's avatar
      sdsformatip() removed. · bbf0736c
      antirez authored
      Specialized single-use function. Not the best match for sds.c btw.
      Also genClientPeerId() is no longer static: we need symbols.
      bbf0736c
    • antirez's avatar
      AnetFormatIP(): renamed, commented, now sticks to IP:port format. · ce269ad3
      antirez authored
      A few code style changes + consistent format: not nice for humans but
      better for parsers.
      ce269ad3
    • Matt Stancliff's avatar
      Sentinel: Improve INFO command behavior · 391fc9b6
      Matt Stancliff authored
      Improvements:
        - Return empty string if asking for non-existing section (INFO foo)
        - Fix potential memory leak (caused by sdsempty() then returned if >2 args)
        - Clean up argument parsing
        - Allow "all" as valid section (same as "default" or zero args currently)
        - Move strcasecmp to end of evaluation chain in conditionals
      
      Also, since we're C99, I moved some variable declarations to be closer
      to where they are actually used (saves us from needing to free an empty info
      if detect argument errors up front).
      
      Closes #1915
      Closes #1966
      391fc9b6