1. 01 Sep, 2020 1 commit
  2. 20 Jul, 2020 3 commits
    • Yossi Gottlieb's avatar
      TLS: Session caching configuration support. (#7420) · 7a536c29
      Yossi Gottlieb authored
      * TLS: Session caching configuration support.
      * TLS: Remove redundant config initialization.
      
      (cherry picked from commit 3e6f2b1a)
      7a536c29
    • Oran Agra's avatar
      RESTORE ABSTTL won't store expired keys into the db (#7472) · 95ba01b5
      Oran Agra authored
      Similarly to EXPIREAT with TTL in the past, which implicitly deletes the
      key and return success, RESTORE should not store key that are already
      expired into the db.
      When used together with REPLACE it should emit a DEL to keyspace
      notification and replication stream.
      
      (cherry picked from commit 5977a948)
      95ba01b5
    • Oran Agra's avatar
      EXEC always fails with EXECABORT and multi-state is cleared · 05e483cb
      Oran Agra authored
      In order to support the use of multi-exec in pipeline, it is important that
      MULTI and EXEC are never rejected and it is easy for the client to know if the
      connection is still in multi state.
      
      It was easy to make sure MULTI and DISCARD never fail (done by previous
      commits) since these only change the client state and don't do any actual
      change in the server, but EXEC is a different story.
      
      Since in the past, it was possible for clients to handle some EXEC errors and
      retry the EXEC, we now can't affort to return any error on EXEC other than
      EXECABORT, which now carries with it the real reason for the abort too.
      
      Other fixes in this commit:
      - Some checks that where performed at the time of queuing need to be re-
        validated when EXEC runs, for instance if the transaction contains writes
        commands, it needs to be aborted. there was one check that was already done
        in execCommand (-READONLY), but other checks where missing: -OOM, -MISCONF,
        -NOREPLICAS, -MASTERDOWN
      - When a command is rejected by processCommand it was rejected with addReply,
        which was not recognized as an error in case the bad command came from the
        master. this will enable to count or MONITOR these errors in the future.
      - make it easier for tests to create additional (non deferred) clients.
      - add tests for the fixes of this commit.
      
      (cherry picked from commit 65a3307b)
      05e483cb
  3. 12 Jun, 2020 2 commits
  4. 28 May, 2020 2 commits
  5. 25 May, 2020 1 commit
    • antirez's avatar
      Make disconnectSlaves() synchronous in the base case. · e3f864b5
      antirez authored
      Otherwise we run into that:
      
      Backtrace:
      src/redis-server 127.0.0.1:21322(logStackTrace+0x45)[0x479035]
      src/redis-server 127.0.0.1:21322(sigsegvHandler+0xb9)[0x4797f9]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fd373c5e390]
      src/redis-server 127.0.0.1:21322(_serverAssert+0x6a)[0x47660a]
      src/redis-server 127.0.0.1:21322(freeReplicationBacklog+0x42)[0x451282]
      src/redis-server 127.0.0.1:21322[0x4552d4]
      src/redis-server 127.0.0.1:21322[0x4c5593]
      src/redis-server 127.0.0.1:21322(aeProcessEvents+0x2e6)[0x42e786]
      src/redis-server 127.0.0.1:21322(aeMain+0x1d)[0x42eb0d]
      src/redis-server 127.0.0.1:21322(main+0x4c5)[0x42b145]
      /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fd3738a3830]
      src/redis-server 127.0.0.1:21322(_start+0x29)[0x42b409]
      
      Since we disconnect all the replicas and free the replication backlog in
      certain replication paths, and the code that will free the replication
      backlog expects that no replica is connected.
      
      However we still need to free the replicas asynchronously in certain
      cases, as documented in the top comment of disconnectSlaves().
      e3f864b5
  6. 15 May, 2020 1 commit
    • antirez's avatar
      Cache master without checking of deferred close flags. · 80c906bd
      antirez authored
      The context is issue #7205: since the introduction of threaded I/O we close
      clients asynchronously by default from readQueryFromClient(). So we
      should no longer prevent the caching of the master client, to later
      PSYNC incrementally, if such flags are set. However we also don't want
      the master client to be cached with such flags (would be closed
      immediately after being restored). And yet we want a way to understand
      if a master was closed because of a protocol error, and in that case
      prevent the caching.
      80c906bd
  7. 14 May, 2020 1 commit
  8. 08 May, 2020 3 commits
  9. 27 Apr, 2020 1 commit
    • Oran Agra's avatar
      Keep track of meaningful replication offset in replicas too · e4d2bb62
      Oran Agra authored
      Now both master and replicas keep track of the last replication offset
      that contains meaningful data (ignoring the tailing pings), and both
      trim that tail from the replication backlog, and the offset with which
      they try to use for psync.
      
      the implication is that if someone missed some pings, or even have
      excessive pings that the promoted replica has, it'll still be able to
      psync (avoid full sync).
      
      the downside (which was already committed) is that replicas running old
      code may fail to psync, since the promoted replica trims pings form it's
      backlog.
      
      This commit adds a test that reproduces several cases of promotions and
      demotions with stale and non-stale pings
      
      Background:
      The mearningful offset on the master was added recently to solve a problem were
      the master is left all alone, injecting PINGs into it's backlog when no one is
      listening and then gets demoted and tries to replicate from a replica that didn't
      have any of the PINGs (or at least not the last ones).
      
      however, consider this case:
      master A has two replicas (B and C) replicating directly from it.
      there's no traffic at all, and also no network issues, just many pings in the
      tail of the backlog. now B gets promoted, A becomes a replica of B, and C
      remains a replica of A. when A gets demoted, it trims the pings from its
      backlog, and successfully replicate from B. however, C is still aware of
      these PINGs, when it'll disconnect and re-connect to A, it'll ask for something
      that's not in the backlog anymore (since A trimmed the tail of it's backlog),
      and be forced to do a full sync (something it didn't have to do before the
      meaningful offset fix).
      
      Besides that, the psync2 test was always failing randomly here and there, it
      turns out the reason were PINGs. Investigating it shows the following scenario:
      
      cycle 1: redis #1 is master, and all the rest are direct replicas of #1
      cycle 2: redis #2 is promoted to master, #1 is a replica of #2 and #3 is replica of #1
      now we see that when #1 is demoted it prints:
      17339:S 21 Apr 2020 11:16:38.523 * Using the meaningful offset 3929963 instead of 3929977 to exclude the final PINGs (14 bytes difference)
      17339:S 21 Apr 2020 11:16:39.391 * Trying a partial resynchronization (request e2b3f8817735fdfe5fa4626766daa938b61419e5:3929964).
      17339:S 21 Apr 2020 11:16:39.392 * Successful partial resynchronization with master.
      and when #3 connects to the demoted #2, #2 says:
      17339:S 21 Apr 2020 11:16:40.084 * Partial resynchronization not accepted: Requested offset for secondary ID was 3929978, but I can reply up to 3929964
      
      so the issue here is that the meaningful offset feature saved the day for the
      demoted master (since it needs to sync from a replica that didn't get the last
      ping), but it didn't help one of the other replicas which did get the last ping.
      e4d2bb62
  10. 24 Apr, 2020 2 commits
    • antirez's avatar
      LCS -> STRALGO LCS. · 3722f89f
      antirez authored
      STRALGO should be a container for mostly read-only string
      algorithms in Redis. The algorithms should have two main
      characteristics:
      
      1. They should be non trivial to compute, and often not part of
      programming language standard libraries.
      2. They should be fast enough that it is a good idea to have optimized C
      implementations.
      
      Next thing I would love to see? A small strings compression algorithm.
      3722f89f
    • antirez's avatar
      Tracking: NOLOOP internals implementation. · 6791ff05
      antirez authored
      6791ff05
  11. 15 Apr, 2020 4 commits
  12. 07 Apr, 2020 5 commits
  13. 31 Mar, 2020 9 commits
  14. 25 Mar, 2020 4 commits
    • antirez's avatar
      PSYNC2: meaningful offset implemented. · 5f72f696
      antirez authored
      A very commonly signaled operational problem with Redis master-replicas
      sets is that, once the master becomes unavailable for some reason,
      especially because of network problems, many times it wont be able to
      perform a partial resynchronization with the new master, once it rejoins
      the partition, for the following reason:
      
      1. The master becomes isolated, however it keeps sending PINGs to the
      replicas. Such PINGs will never be received since the link connection is
      actually already severed.
      2. On the other side, one of the replicas will turn into the new master,
      setting its secondary replication ID offset to the one of the last
      command received from the old master: this offset will not include the
      PINGs sent by the master once the link was already disconnected.
      3. When the master rejoins the partion and is turned into a replica, its
      offset will be too advanced because of the PINGs, so a PSYNC will fail,
      and a full synchronization will be required.
      
      Related to issue #7002 and other discussion we had in the past around
      this problem.
      5f72f696
    • bodong.ybd's avatar
      b3e4abf0
    • WuYunlong's avatar
      Fix master replica inconsistency for upgrading scenario. · 0578157d
      WuYunlong authored
      Before this commit, when upgrading a replica, expired keys will not
      be loaded, thus causing replica having less keys in db. To this point,
      master and replica's keys is logically consistent. However, before
      the keys in master and replica are physically consistent, that is,
      they have the same dbsize, if master got a problem and the replica
      got promoted and becomes new master of that partition, and master
      updates a key which does not exist on master, but physically exists
      on the old master(new replica), the old master would refuse to update
      the key, thus causing master and replica data inconsistent.
      
      How could this happen?
      That's all because of the wrong judgement of roles while starting up
      the server. We can not use server.masterhost to judge if the server
      is master or replica, since it fails in cluster mode.
      
      When we start the server, we load rdb and do want to load expired keys,
      and do not want to have the ability to active expire keys, if it is
      a replica.
      0578157d
    • antirez's avatar
      ACL: Make Redis 6 more backward compatible with requirepass. · 9c2e42dd
      antirez authored
      Note that this as a side effect fixes Sentinel "requirepass" mode.
      9c2e42dd
  15. 05 Mar, 2020 1 commit