1. 03 Jun, 2020 1 commit
  2. 02 May, 2020 1 commit
    • zhenwei pi's avatar
      Support setcpuaffinity on linux/bsd · 1a0deab2
      zhenwei pi authored
      Currently, there are several types of threads/child processes of a
      redis server. Sometimes we need deeply optimise the performance of
      redis, so we would like to isolate threads/processes.
      
      There were some discussion about cpu affinity cases in the issue:
      https://github.com/antirez/redis/issues/2863
      
      
      
      So implement cpu affinity setting by redis.conf in this patch, then
      we can config server_cpulist/bio_cpulist/aof_rewrite_cpulist/
      bgsave_cpulist by cpu list.
      
      Examples of cpulist in redis.conf:
      server_cpulist 0-7:2      means cpu affinity 0,2,4,6
      bio_cpulist 1,3           means cpu affinity 1,3
      aof_rewrite_cpulist 8-11  means cpu affinity 8,9,10,11
      bgsave_cpulist 1,10-11    means cpu affinity 1,10,11
      
      Test on linux/freebsd, both work fine.
      Signed-off-by: default avatarzhenwei pi <pizhenwei@bytedance.com>
      1a0deab2
  3. 31 Mar, 2020 1 commit
  4. 25 Mar, 2020 1 commit
  5. 06 Feb, 2020 1 commit
    • Oran Agra's avatar
      stopAppendOnly resets aof_rewrite_scheduled · 69e8ea71
      Oran Agra authored
      althouh in theory, users can do BGREWRITEAOF even if aof is disabled, i
      suppose it is more common that the scheduled flag is set by either
      startAppendOnly, of a failed initial AOFRW fork (AOF_WAIT_REWRITE)
      69e8ea71
  6. 17 Jan, 2020 1 commit
  7. 13 Jan, 2020 2 commits
  8. 10 Jan, 2020 1 commit
  9. 06 Jan, 2020 1 commit
  10. 29 Oct, 2019 1 commit
    • Oran Agra's avatar
      Modules hooks: complete missing hooks for the initial set of hooks · 51c3ff8d
      Oran Agra authored
      * replication hooks: role change, master link status, replica online/offline
      * persistence hooks: saving, loading, loading progress
      * misc hooks: cron loop, shutdown, module loaded/unloaded
      * change the way hooks test work, and add tests for all of the above
      
      startLoading() now gets flag indicating what is loaded.
      stopLoading() now gets an indication of success or failure.
      adding startSaving() and stopSaving() with similar args and role.
      51c3ff8d
  11. 17 Oct, 2019 1 commit
  12. 07 Oct, 2019 3 commits
    • Oran Agra's avatar
      TLS: Implement support for write barrier. · 6b629480
      Oran Agra authored
      6b629480
    • Oran Agra's avatar
      diskless replication rdb transfer uses pipe, and writes to sockets form the parent process. · 5a477946
      Oran Agra authored
      misc:
      - handle SSL_has_pending by iterating though these in beforeSleep, and setting timeout of 0 to aeProcessEvents
      - fix issue with epoll signaling EPOLLHUP and EPOLLERR only to the write handlers. (needed to detect the rdb pipe was closed)
      - add key-load-delay config for testing
      - trim connShutdown which is no longer needed
      - rioFdsetWrite -> rioFdWrite - simplified since there's no longer need to write to multiple FDs
      - don't detect rdb child exited (don't call wait3) until we detect the pipe is closed
      - Cleanup bad optimization from rio.c, add another one
      5a477946
    • Yossi Gottlieb's avatar
      TLS: Connections refactoring and TLS support. · b087dd1d
      Yossi Gottlieb authored
      * Introduce a connection abstraction layer for all socket operations and
      integrate it across the code base.
      * Provide an optional TLS connections implementation based on OpenSSL.
      * Pull a newer version of hiredis with TLS support.
      * Tests, redis-cli updates for TLS support.
      b087dd1d
  13. 27 Sep, 2019 1 commit
  14. 26 Sep, 2019 1 commit
  15. 04 Sep, 2019 1 commit
  16. 17 Jul, 2019 1 commit
    • Oran Agra's avatar
      Module API for Forking · 56258c6b
      Oran Agra authored
      * create module API for forking child processes.
      * refactor duplicate code around creating and tracking forks by AOF and RDB.
      * child processes listen to SIGUSR1 and dies exitFromChild in order to
        eliminate a valgrind warning of unhandled signal.
      * note that BGSAVE error reply has changed.
      
      valgrind error is:
        Process terminating with default action of signal 10 (SIGUSR1)
      56258c6b
  17. 12 Jul, 2019 1 commit
  18. 08 Jul, 2019 1 commit
    • Oran Agra's avatar
      diskless replication on slave side (don't store rdb to file), plus some other related fixes · 2de544cf
      Oran Agra authored
      The implementation of the diskless replication was currently diskless only on the master side.
      The slave side was still storing the received rdb file to the disk before loading it back in and parsing it.
      
      This commit adds two modes to load rdb directly from socket:
      1) when-empty
      2) using "swapdb"
      the third mode of using diskless slave by flushdb is risky and currently not included.
      
      other changes:
      --------------
      distinguish between aof configuration and state so that we can re-enable aof only when sync eventually
      succeeds (and not when exiting from readSyncBulkPayload after a failed attempt)
      also a CONFIG GET and INFO during rdb loading would have lied
      
      When loading rdb from the network, don't kill the server on short read (that can be a network error)
      
      Fix rdb check when performed on preamble AOF
      
      tests:
      run replication tests for diskless slave too
      make replication test a bit more aggressive
      Add test for diskless load swapdb
      2de544cf
  19. 29 May, 2019 1 commit
  20. 29 Apr, 2019 1 commit
  21. 15 Mar, 2019 1 commit
  22. 12 Mar, 2019 1 commit
  23. 21 Jan, 2019 4 commits
  24. 14 Jan, 2019 1 commit
  25. 16 Oct, 2018 2 commits
  26. 09 Oct, 2018 2 commits
  27. 03 Aug, 2018 2 commits
  28. 16 Jul, 2018 1 commit
    • Oran Agra's avatar
      slave buffers were wasteful and incorrectly counted causing eviction · bf680b6f
      Oran Agra authored
      A) slave buffers didn't count internal fragmentation and sds unused space,
         this caused them to induce eviction although we didn't mean for it.
      
      B) slave buffers were consuming about twice the memory of what they actually needed.
      - this was mainly due to sdsMakeRoomFor growing to twice as much as needed each time
        but networking.c not storing more than 16k (partially fixed recently in 237a38737).
      - besides it wasn't able to store half of the new string into one buffer and the
        other half into the next (so the above mentioned fix helped mainly for small items).
      - lastly, the sds buffers had up to 30% internal fragmentation that was wasted,
        consumed but not used.
      
      C) inefficient performance due to starting from a small string and reallocing many times.
      
      what i changed:
      - creating dedicated buffers for reply list, counting their size with zmalloc_size
      - when creating a new reply node from, preallocate it to at least 16k.
      - when appending a new reply to the buffer, first fill all the unused space of the
        previous node before starting a new one.
      
      other changes:
      - expose mem_not_counted_for_evict info field for the benefit of the test suite
      - add a test to make sure slave buffers are counted correctly and that they don't cause eviction
      bf680b6f
  29. 03 Jul, 2018 1 commit
  30. 19 Jun, 2018 2 commits
    • antirez's avatar
      AOF: remove no longer used variable "now". · 333c98c4
      antirez authored
      333c98c4
    • antirez's avatar
      Remove AOF optimization to skip expired keys. · ba92b517
      antirez authored
      Basically we cannot be sure that if the key is expired while writing the
      AOF, the main thread will surely find the key expired. There are
      possible race conditions like the moment at which the "now" is sampled,
      and the fact that time may jump backward.
      
      Think about the following:
      
      SET a 5
      EXPIRE a 1
      
      AOF rewrite starts after about 1 second. The child process finds the key
      expired, while in the main thread instead an INCR command is called
      against the key "a" immediately after a fork, and the scheduler was
      faster to give execution time to the main thread, so "a" is yet not
      expired.
      
      The main thread will generate an INCR a command to the AOF log that will
      be appended to the rewritten AOF file, but that INCR command will target
      a non existin "a" key, so a new non volatile key "a" will be created.
      
      Two observations:
      
      A) In theory by computing "now" before the fork, we should be sure that
      if a key is expired at that time, it will be expired later when the
      main thread will try to access to such key. However this does not take
      into account the fact that the computer time may jump backward.
      
      B) Technically we may still make the process safe by using a monotonic
      time source.
      
      However there were other similar related bugs, and in general the new
      "vision" is that Redis persistence files should represent the memory
      state without trying to be too smart: this makes the design more
      consistent, bugs less likely to arise from complex interactions, and in
      the end what is to fix is the Redis expire process to have less expired
      keys in RAM.
      
      Thanks to Oran Agra and Guy Benoish for writing me an email outlining
      this problem, after they conducted a Redis 5 code review.
      ba92b517