1. 14 May, 2020 1 commit
  2. 12 May, 2020 1 commit
  3. 05 May, 2020 1 commit
  4. 02 May, 2020 2 commits
    • zhenwei pi's avatar
      Support setcpuaffinity on linux/bsd · 1a0deab2
      zhenwei pi authored
      Currently, there are several types of threads/child processes of a
      redis server. Sometimes we need deeply optimise the performance of
      redis, so we would like to isolate threads/processes.
      
      There were some discussion about cpu affinity cases in the issue:
      https://github.com/antirez/redis/issues/2863
      
      
      
      So implement cpu affinity setting by redis.conf in this patch, then
      we can config server_cpulist/bio_cpulist/aof_rewrite_cpulist/
      bgsave_cpulist by cpu list.
      
      Examples of cpulist in redis.conf:
      server_cpulist 0-7:2      means cpu affinity 0,2,4,6
      bio_cpulist 1,3           means cpu affinity 1,3
      aof_rewrite_cpulist 8-11  means cpu affinity 8,9,10,11
      bgsave_cpulist 1,10-11    means cpu affinity 1,10,11
      
      Test on linux/freebsd, both work fine.
      Signed-off-by: default avatarzhenwei pi <pizhenwei@bytedance.com>
      1a0deab2
    • Oran Agra's avatar
      optimize memory usage of deferred replies - fixed · 6726b3c2
      Oran Agra authored
      When deffered reply is added the previous reply node cannot be used so
      all the extra space we allocated in it is wasted. in case someone uses
      deffered replies in a loop, each time adding a small reply, each of
      these reply nodes (the small string reply) would have consumed a 16k
      block.
      now when we add anther diferred reply node, we trim the unused portion
      of the previous reply block.
      
      see #7123
      
      cherry picked from commit fb732f7a
      with fix to handle a crash with LIBC allocator, which apparently can
      return the same pointer despite changing it's size.
      i.e. shrinking an allocation of 16k into 56 bytes without changing the
      pointer.
      6726b3c2
  5. 01 May, 2020 2 commits
  6. 30 Apr, 2020 1 commit
  7. 27 Apr, 2020 1 commit
    • Oran Agra's avatar
      Keep track of meaningful replication offset in replicas too · 4447ddc8
      Oran Agra authored
      Now both master and replicas keep track of the last replication offset
      that contains meaningful data (ignoring the tailing pings), and both
      trim that tail from the replication backlog, and the offset with which
      they try to use for psync.
      
      the implication is that if someone missed some pings, or even have
      excessive pings that the promoted replica has, it'll still be able to
      psync (avoid full sync).
      
      the downside (which was already committed) is that replicas running old
      code may fail to psync, since the promoted replica trims pings form it's
      backlog.
      
      This commit adds a test that reproduces several cases of promotions and
      demotions with stale and non-stale pings
      
      Background:
      The mearningful offset on the master was added recently to solve a problem were
      the master is left all alone, injecting PINGs into it's backlog when no one is
      listening and then gets demoted and tries to replicate from a replica that didn't
      have any of the PINGs (or at least not the last ones).
      
      however, consider this case:
      master A has two replicas (B and C) replicating directly from it.
      there's no traffic at all, and also no network issues, just many pings in the
      tail of the backlog. now B gets promoted, A becomes a replica of B, and C
      remains a replica of A. when A gets demoted, it trims the pings from its
      backlog, and successfully replicate from B. however, C is still aware of
      these PINGs, when it'll disconnect and re-connect to A, it'll ask for something
      that's not in the backlog anymore (since A trimmed the tail of it's backlog),
      and be forced to do a full sync (something it didn't have to do before the
      meaningful offset fix).
      
      Besides that, the psync2 test was always failing randomly here and there, it
      turns out the reason were PINGs. Investigating it shows the following scenario:
      
      cycle 1: redis #1 is master, and all the rest are direct replicas of #1
      cycle 2: redis #2 is promoted to master, #1 is a replica of #2 and #3 is replica of #1
      now we see that when #1 is demoted it prints:
      17339:S 21 Apr 2020 11:16:38.523 * Using the meaningful offset 3929963 instead of 3929977 to exclude the final PINGs (14 bytes difference)
      17339:S 21 Apr 2020 11:16:39.391 * Trying a partial resynchronization (request e2b3f8817735fdfe5fa4626766daa938b61419e5:3929964).
      17339:S 21 Apr 2020 11:16:39.392 * Successful partial resynchronization with master.
      and when #3 connects to the demoted #2, #2 says:
      17339:S 21 Apr 2020 11:16:40.084 * Partial resynchronization not accepted: Requested offset for secondary ID was 3929978, but I can reply up to 3929964
      
      so the issue here is that the meaningful offset feature saved the day for the
      demoted master (since it needs to sync from a replica that didn't get the last
      ping), but it didn't help one of the other replicas which did get the last ping.
      4447ddc8
  8. 24 Apr, 2020 1 commit
    • Oran Agra's avatar
      optimize memory usage of deferred replies · fb732f7a
      Oran Agra authored
      When deffered reply is added the previous reply node cannot be used so
      all the extra space we allocated in it is wasted. in case someone uses
      deffered replies in a loop, each time adding a small reply, each of
      these reply nodes (the small string reply) would have consumed a 16k
      block.
      now when we add anther diferred reply node, we trim the unused portion
      of the previous reply block.
      
      see #7123
      fb732f7a
  9. 21 Apr, 2020 1 commit
  10. 18 Apr, 2020 1 commit
    • zhenwei pi's avatar
      Threaded IO: set thread name for redis-server · 5010da6a
      zhenwei pi authored
      
      
      Set thread name for each thread of redis-server, this helps us to
      monitor the utilization and optimise the performance.
      
      And suggested-by Salvatore, implement this feature for multi
      platforms. Currently support linux and bsd, ignore other OS.
      
      An exmaple on Linux:
       # top -d 5 -p `pidof redis-server ` -H
      
          PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
      3682671 root      20   0  227744   8248   3836 R 99.2  0.0   0:19.53 redis-server
      3682677 root      20   0  227744   8248   3836 S 26.4  0.0   0:04.15 io_thd_3
      3682675 root      20   0  227744   8248   3836 S 23.6  0.0   0:03.98 io_thd_1
      3682676 root      20   0  227744   8248   3836 S 23.6  0.0   0:03.97 io_thd_2
      3682672 root      20   0  227744   8248   3836 S  0.2  0.0   0:00.02 bio_close_file
      3682673 root      20   0  227744   8248   3836 S  0.2  0.0   0:00.02 bio_aof_fsync
      3682674 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 bio_lazy_free
      3682678 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
      3682682 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
      3682683 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
      3682684 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
      3682685 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
      3682687 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
      
      Another exmaple on FreeBSD-12.1:
        PID USERNAME    PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
       5212 root        100    0    48M  7280K CPU2     2   0:26  99.52% redis-server{redis-server}
       5212 root         38    0    48M  7280K umtxn    4   0:06  26.94% redis-server{io_thd_3}
       5212 root         36    0    48M  7280K umtxn    6   0:06  26.84% redis-server{io_thd_1}
       5212 root         39    0    48M  7280K umtxn    1   0:06  25.30% redis-server{io_thd_2}
       5212 root         20    0    48M  7280K uwait    3   0:00   0.00% redis-server{redis-server}
       5212 root         21    0    48M  7280K uwait    2   0:00   0.00% redis-server{bio_close_file}
       5212 root         21    0    48M  7280K uwait    3   0:00   0.00% redis-server{bio_aof_fsync}
       5212 root         21    0    48M  7280K uwait    0   0:00   0.00% redis-server{bio_lazy_free}
      Signed-off-by: default avatarzhenwei pi <pizhenwei@bytedance.com>
      5010da6a
  11. 16 Apr, 2020 1 commit
  12. 15 Apr, 2020 1 commit
  13. 07 Apr, 2020 1 commit
    • antirez's avatar
      Speedup INFO by counting client memory incrementally. · f6987628
      antirez authored
      Related to #5145.
      
      Design note: clients may change type when they turn into replicas or are
      moved into the Pub/Sub category and so forth. Moreover the recomputation
      of the bytes used is problematic for obvious reasons: it changes
      continuously, so as a conservative way to avoid accumulating errors,
      each client remembers the contribution it gave to the sum, and removes
      it when it is freed or before updating it with the new memory usage.
      f6987628
  14. 31 Mar, 2020 2 commits
  15. 30 Mar, 2020 2 commits
  16. 22 Mar, 2020 1 commit
  17. 20 Mar, 2020 1 commit
  18. 15 Mar, 2020 3 commits
  19. 24 Feb, 2020 1 commit
  20. 16 Feb, 2020 2 commits
  21. 14 Feb, 2020 1 commit
  22. 13 Feb, 2020 1 commit
  23. 12 Feb, 2020 1 commit
  24. 11 Feb, 2020 1 commit
  25. 10 Feb, 2020 1 commit
  26. 06 Feb, 2020 3 commits
  27. 13 Jan, 2020 3 commits
  28. 29 Dec, 2019 2 commits
    • antirez's avatar
      Fix duplicated CLIENT SETNAME reply. · 6e4f70b8
      antirez authored
      Happened when we set the name to "" to cancel the name.
      Was introduced during the RESP3 refactoring.
      
      See #6036.
      6e4f70b8
    • antirez's avatar
      Inline protocol: handle empty strings well. · 5521910d
      antirez authored
      This bug is from the first version of Redis. Probably the problem here
      is that before we used an SDS split function that created empty strings
      for additional spaces, like in "SET    foo          bar".
      AFAIK later we replaced it with the curretn sdssplitarg() API that has
      no such a problem. As a result, we introduced a bug, where it is no
      longer possible to do something like:
      
          SET foo ""
      
      Using the inline protocol. Now it is fixed.
      5521910d