1. 26 Sep, 2019 1 commit
  2. 04 Sep, 2019 1 commit
  3. 12 Jul, 2019 1 commit
  4. 08 Jul, 2019 1 commit
    • Oran Agra's avatar
      diskless replication on slave side (don't store rdb to file), plus some other related fixes · 2de544cf
      Oran Agra authored
      The implementation of the diskless replication was currently diskless only on the master side.
      The slave side was still storing the received rdb file to the disk before loading it back in and parsing it.
      
      This commit adds two modes to load rdb directly from socket:
      1) when-empty
      2) using "swapdb"
      the third mode of using diskless slave by flushdb is risky and currently not included.
      
      other changes:
      --------------
      distinguish between aof configuration and state so that we can re-enable aof only when sync eventually
      succeeds (and not when exiting from readSyncBulkPayload after a failed attempt)
      also a CONFIG GET and INFO during rdb loading would have lied
      
      When loading rdb from the network, don't kill the server on short read (that can be a network error)
      
      Fix rdb check when performed on preamble AOF
      
      tests:
      run replication tests for diskless slave too
      make replication test a bit more aggressive
      Add test for diskless load swapdb
      2de544cf
  5. 29 Apr, 2019 1 commit
  6. 15 Mar, 2019 1 commit
  7. 12 Mar, 2019 1 commit
  8. 21 Jan, 2019 4 commits
  9. 14 Jan, 2019 1 commit
  10. 16 Oct, 2018 2 commits
  11. 09 Oct, 2018 2 commits
  12. 03 Aug, 2018 2 commits
  13. 16 Jul, 2018 1 commit
    • Oran Agra's avatar
      slave buffers were wasteful and incorrectly counted causing eviction · bf680b6f
      Oran Agra authored
      A) slave buffers didn't count internal fragmentation and sds unused space,
         this caused them to induce eviction although we didn't mean for it.
      
      B) slave buffers were consuming about twice the memory of what they actually needed.
      - this was mainly due to sdsMakeRoomFor growing to twice as much as needed each time
        but networking.c not storing more than 16k (partially fixed recently in 237a38737).
      - besides it wasn't able to store half of the new string into one buffer and the
        other half into the next (so the above mentioned fix helped mainly for small items).
      - lastly, the sds buffers had up to 30% internal fragmentation that was wasted,
        consumed but not used.
      
      C) inefficient performance due to starting from a small string and reallocing many times.
      
      what i changed:
      - creating dedicated buffers for reply list, counting their size with zmalloc_size
      - when creating a new reply node from, preallocate it to at least 16k.
      - when appending a new reply to the buffer, first fill all the unused space of the
        previous node before starting a new one.
      
      other changes:
      - expose mem_not_counted_for_evict info field for the benefit of the test suite
      - add a test to make sure slave buffers are counted correctly and that they don't cause eviction
      bf680b6f
  14. 03 Jul, 2018 1 commit
  15. 19 Jun, 2018 2 commits
    • antirez's avatar
      AOF: remove no longer used variable "now". · 333c98c4
      antirez authored
      333c98c4
    • antirez's avatar
      Remove AOF optimization to skip expired keys. · ba92b517
      antirez authored
      Basically we cannot be sure that if the key is expired while writing the
      AOF, the main thread will surely find the key expired. There are
      possible race conditions like the moment at which the "now" is sampled,
      and the fact that time may jump backward.
      
      Think about the following:
      
      SET a 5
      EXPIRE a 1
      
      AOF rewrite starts after about 1 second. The child process finds the key
      expired, while in the main thread instead an INCR command is called
      against the key "a" immediately after a fork, and the scheduler was
      faster to give execution time to the main thread, so "a" is yet not
      expired.
      
      The main thread will generate an INCR a command to the AOF log that will
      be appended to the rewritten AOF file, but that INCR command will target
      a non existin "a" key, so a new non volatile key "a" will be created.
      
      Two observations:
      
      A) In theory by computing "now" before the fork, we should be sure that
      if a key is expired at that time, it will be expired later when the
      main thread will try to access to such key. However this does not take
      into account the fact that the computer time may jump backward.
      
      B) Technically we may still make the process safe by using a monotonic
      time source.
      
      However there were other similar related bugs, and in general the new
      "vision" is that Redis persistence files should represent the memory
      state without trying to be too smart: this makes the design more
      consistent, bugs less likely to arise from complex interactions, and in
      the end what is to fix is the Redis expire process to have less expired
      keys in RAM.
      
      Thanks to Oran Agra and Guy Benoish for writing me an email outlining
      this problem, after they conducted a Redis 5 code review.
      ba92b517
  16. 29 May, 2018 1 commit
    • antirez's avatar
      Don't expire keys while loading RDB from AOF preamble. · 49147f36
      antirez authored
      The AOF tail of a combined RDB+AOF is based on the premise of applying
      the AOF commands to the exact state that there was in the server while
      the RDB was persisted. By expiring keys while loading the RDB file, we
      change the state, so applying the AOF tail later may change the state.
      
      Test case:
      
      * Time1: SET a 10
      * Time2: EXPIREAT a $time5
      * Time3: INCR a
      * Time4: PERSIT A. Start bgrewiteaof with RDB preamble. The value of a is 11 without expire time.
      * Time5: Restart redis from the RDB+AOF: consistency violation.
      
      Thanks to @soloestoy for providing the patch.
      Thanks to @trevor211 for the original issue report and the initial fix.
      
      Check issue #4950 for more info.
      49147f36
  17. 23 Mar, 2018 1 commit
  18. 15 Mar, 2018 1 commit
  19. 13 Feb, 2018 2 commits
    • antirez's avatar
      Make it explicit with a comment why we kill the old AOF rewrite. · c14ba46e
      antirez authored
      See #3858.
      c14ba46e
    • Guy Benoish's avatar
      rewriteAppendOnlyFileBackground() failure fix · f7820067
      Guy Benoish authored
      It is possible to do BGREWRITEAOF even if appendonly=no. This is by design.
      stopAppendonly() didn't turn off aof_rewrite_scheduled (it can be turned on
      again by BGREWRITEAOF even while appendonly is off anyway).
      After configuring `appendonly yes` it will see that the state is AOF_OFF,
      there's no RDB fork, so it will do rewriteAppendOnlyFileBackground() which
      will fail since the aof_child_pid is set (was scheduled and started by cron).
      
      Solution:
      stopAppendonly() will turn off the schedule flag (regardless of who asked for it).
      startAppendonly() will terminate any existing fork and start a new one (so it is the most recent).
      f7820067
  20. 15 Jan, 2018 1 commit
  21. 15 Dec, 2017 1 commit
  22. 14 Dec, 2017 1 commit
  23. 01 Dec, 2017 2 commits
  24. 30 Nov, 2017 2 commits
  25. 22 Jun, 2017 2 commits
  26. 16 Jun, 2017 1 commit
  27. 18 Apr, 2017 1 commit
  28. 23 Feb, 2017 1 commit
  29. 20 Feb, 2017 1 commit