1. 21 Jul, 2021 1 commit
    • Oran Agra's avatar
      if diskless repl child is killed, make sure to reap the pid (#7742) · 18429ce5
      Oran Agra authored
      Starting redis 6.0 and the changes we made to the diskless master to be
      suitable for TLS, I made the master avoid reaping (wait3) the pid of the
      child until we know all replicas are done reading their rdb.
      
      I did that in order to avoid a state where the rdb_child_pid is -1 but
      we don't yet want to start another fork (still busy serving that data to
      replicas).
      
      It turns out that the solution used so far was problematic in case the
      fork child was being killed (e.g. by the kernel OOM killer), in that
      case there's a chance that we currently disabled the read event on the
      rdb pipe, since we're waiting for a replica to become writable again.
      and in that scenario the master would have never realized the child
      exited, and the replica will remain hung too.
      Note that there's no mechanism to detect a hung replica while it's in
      rdb transfer state.
      
      The solution here is to add another pipe which is used by the parent to
      tell the child it is safe to exit. this mean that when the child exits,
      for whatever reason, it is safe to reap it.
      
      Besides that, i'm re-introducing an adjustment to REPLCONF ACK which was
      part of #6271 (Accelerate diskless master connections) but was dropped
      when that PR was rebased after the TLS fork/pipe changes (5a477946).
      Now that RdbPipeCleanup no longer calls checkChildrenDone, and the ACK
      has chance to detect that the child exited, it should be the one to call
      it so that we don't have to wait for cron (server.hz) to do that.
      
      (cherry picked from commit 573246f7)
      18429ce5
  2. 12 Jan, 2021 2 commits
  3. 27 Oct, 2020 7 commits
    • WuYunlong's avatar
      Update rdb_last_bgsave_time_sec in INFO on diskless replication (#7917) · 6f16e52a
      WuYunlong authored
      `info Persistence` will include correct (updated) rdb_last_bgsave_time_sec
      For diskless bgsave (sockets) too (like a few other persistence info fields).
      
      Refactor code to reduce duplicate code.
      
      (cherry picked from commit e05a7df7)
      6f16e52a
    • Wen Hui's avatar
      rdb.c: handle fclose error case differently to avoid double fclose (#7307) · faa431c9
      Wen Hui authored
      When fclose would fail, the previous implementation would have attempted to do fclose again
      this can in theory lead to segfault.
      
      other changes:
      check for non-zero return value as failure rather than a specific error code.
      this doesn't fix a real bug, just a minor cleanup.
      
      (cherry picked from commit 323029ba)
      faa431c9
    • Ariel Shtul's avatar
      Fix redis-check-rdb support for modules aux data (#7826) · 6a4da495
      Ariel Shtul authored
      
      
      redis-check-rdb was unable to parse rdb files containing module aux data.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      (cherry picked from commit 63a05dde)
      6a4da495
    • Oran Agra's avatar
      RM_GetContextFlags provides indication that we're in a fork child (#7783) · fd610ae0
      Oran Agra authored
      
      (cherry picked from commit 2458e548)
      fd610ae0
    • Wang Yuan's avatar
      Remove tmp rdb file in background thread (#7762) · 0bdddd3c
      Wang Yuan authored
      We're already using bg_unlink in several places to delete the rdb file in the background,
      and avoid paying the cost of the deletion from our main thread.
      This commit uses bg_unlink to remove the temporary rdb file in the background too.
      
      However, in case we delete that rdb file just before exiting, we don't actually wait for the
      background thread or the main thread to delete it, and just let the OS clean up after us.
      i.e. we open the file, unlink it and exit with the fd still open.
      
      Furthermore, rdbRemoveTempFile can be called from a thread and was using snprintf which is
      not async-signal-safe, we now use ll2string instead.
      
      (cherry picked from commit b002d2b4)
      0bdddd3c
    • Oran Agra's avatar
      Add printf attribute and fix warnings and a minor bug (#7803) · 7bc07260
      Oran Agra authored
      The fix in error handling of rdbGenericLoadStringObject is an actual bugfix
      
      (cherry picked from commit 092cfca5)
      7bc07260
    • Oran Agra's avatar
      Squash merging 125 typo/grammar/comment/doc PRs (#7773) · 2ae7f491
      Oran Agra authored
      List of squashed commits or PRs
      ===============================
      
      commit 66801ea
      Author: hwware <wen.hui.ware@gmail.com>
      Date:   Mon Jan 13 00:54:31 2020 -0500
      
          typo fix in acl.c
      
      commit 46f55db
      Author: Itamar Haber <itamar@redislabs.com>
      Date:   Sun Sep 6 18:24:11 2020 +0300
      
          Updates a couple of comments
      
          Specifically:
      
          * RM_AutoMemory completed instead of pointing to docs
          * Updated link to custom type doc
      
      commit 61a2aa0
      Author: xindoo <xindoo@qq.com>
      Date:   Tue Sep 1 19:24:59 2020 +0800
      
          Correct errors in code comments
      
      commit a5871d1
      Author: yz1509 <pro-756@qq.com>
      Date:   Tue Sep 1 18:36:06 2020 +0800
      
          fix typos in module.c
      
      commit 41eede7
      Author: bookug <bookug@qq.com>
      Date:   Sat Aug 15 01:11:33 2020 +0800
      
          docs: fix typos in comments
      
      commit c303c84
      Author: lazy-snail <ws.niu@outlook.com>
      Date:   Fri Aug 7 11:15:44 2020 +0800
      
          fix spelling in redis.conf
      
      commit 1eb76bf
      Author: zhujian <zhujianxyz@gmail.com>...
      2ae7f491
  4. 01 Sep, 2020 2 commits
  5. 08 May, 2020 3 commits
    • zhenwei pi's avatar
      Support setcpuaffinity on linux/bsd · d6436eb7
      zhenwei pi authored
      Currently, there are several types of threads/child processes of a
      redis server. Sometimes we need deeply optimise the performance of
      redis, so we would like to isolate threads/processes.
      
      There were some discussion about cpu affinity cases in the issue:
      https://github.com/antirez/redis/issues/2863
      
      
      
      So implement cpu affinity setting by redis.conf in this patch, then
      we can config server_cpulist/bio_cpulist/aof_rewrite_cpulist/
      bgsave_cpulist by cpu list.
      
      Examples of cpulist in redis.conf:
      server_cpulist 0-7:2      means cpu affinity 0,2,4,6
      bio_cpulist 1,3           means cpu affinity 1,3
      aof_rewrite_cpulist 8-11  means cpu affinity 8,9,10,11
      bgsave_cpulist 1,10-11    means cpu affinity 1,10,11
      
      Test on linux/freebsd, both work fine.
      Signed-off-by: default avatarzhenwei pi <pizhenwei@bytedance.com>
      d6436eb7
    • Guy Benoish's avatar
      XPENDING should not update consumer's seen-time · 3a441c7d
      Guy Benoish authored
      Same goes for XGROUP DELCONSUMER (But in this case, it doesn't
      have any visible effect)
      3a441c7d
    • Oran Agra's avatar
      add daily github actions with libc malloc and valgrind · 3d3861dd
      Oran Agra authored
      * fix memlry leaks with diskless replica short read.
      * fix a few timing issues with valgrind runs
      * fix issue with valgrind and watchdog schedule signal
      
      about the valgrind WD issue:
      the stack trace test in logging.tcl, has issues with valgrind:
      ==28808== Can't extend stack to 0x1ffeffdb38 during signal delivery for thread 1:
      ==28808==   too small or bad protection modes
      
      it seems to be some valgrind bug with SA_ONSTACK.
      SA_ONSTACK seems unneeded since WD is not recursive (SA_NODEFER was removed),
      also, not sure if it's even valid without a call to sigaltstack()
      3d3861dd
  6. 01 May, 2020 1 commit
    • antirez's avatar
      Cast printf() argument to the format specifier. · db73d099
      antirez authored
      We could use uint64_t specific macros, but after all it's simpler to
      just use an obvious equivalent type plus casting: this will be a no op
      and is simpler than fixed size types printf macros.
      db73d099
  7. 28 Apr, 2020 1 commit
  8. 15 Apr, 2020 4 commits
  9. 07 Apr, 2020 2 commits
  10. 25 Mar, 2020 1 commit
    • WuYunlong's avatar
      Fix master replica inconsistency for upgrading scenario. · 0578157d
      WuYunlong authored
      Before this commit, when upgrading a replica, expired keys will not
      be loaded, thus causing replica having less keys in db. To this point,
      master and replica's keys is logically consistent. However, before
      the keys in master and replica are physically consistent, that is,
      they have the same dbsize, if master got a problem and the replica
      got promoted and becomes new master of that partition, and master
      updates a key which does not exist on master, but physically exists
      on the old master(new replica), the old master would refuse to update
      the key, thus causing master and replica data inconsistent.
      
      How could this happen?
      That's all because of the wrong judgement of roles while starting up
      the server. We can not use server.masterhost to judge if the server
      is master or replica, since it fails in cluster mode.
      
      When we start the server, we load rdb and do want to load expired keys,
      and do not want to have the ability to active expire keys, if it is
      a replica.
      0578157d
  11. 05 Mar, 2020 1 commit
  12. 12 Feb, 2020 1 commit
  13. 10 Nov, 2019 1 commit
    • Oran Agra's avatar
      rename RN_SetLRUOrLFU -> RM_SetLRU and RN_SetLFU · 28c20b4e
      Oran Agra authored
      - the API name was odd, separated to two apis one for LRU and one for LFU
      - the LRU idle time was in 1 second resolution, which might be ok for RDB
        and RESTORE, but i think modules may need higher resolution
      - adding tests for LFU and for handling maxmemory policy mismatch
      28c20b4e
  14. 05 Nov, 2019 1 commit
    • antirez's avatar
      Update PR #6537 patch to for generality. · 824f5f0b
      antirez authored
      After the thread in #6537 and thanks to the suggestions received, this
      commit updates the original patch in order to:
      
      1. Solve the problem of updating the time in multiple places by updating
      it in call().
      2. Avoid introducing a new field but use our cached time.
      
      This required some minor refactoring to the function updating the time,
      and the introduction of a new cached time in microseconds in order to
      use less gettimeofday() calls.
      824f5f0b
  15. 29 Oct, 2019 1 commit
    • Oran Agra's avatar
      Modules hooks: complete missing hooks for the initial set of hooks · 51c3ff8d
      Oran Agra authored
      * replication hooks: role change, master link status, replica online/offline
      * persistence hooks: saving, loading, loading progress
      * misc hooks: cron loop, shutdown, module loaded/unloaded
      * change the way hooks test work, and add tests for all of the above
      
      startLoading() now gets flag indicating what is loaded.
      stopLoading() now gets an indication of success or failure.
      adding startSaving() and stopSaving() with similar args and role.
      51c3ff8d
  16. 24 Oct, 2019 1 commit
  17. 15 Oct, 2019 1 commit
  18. 07 Oct, 2019 2 commits
    • Oran Agra's avatar
      diskless replication rdb transfer uses pipe, and writes to sockets form the parent process. · 5a477946
      Oran Agra authored
      misc:
      - handle SSL_has_pending by iterating though these in beforeSleep, and setting timeout of 0 to aeProcessEvents
      - fix issue with epoll signaling EPOLLHUP and EPOLLERR only to the write handlers. (needed to detect the rdb pipe was closed)
      - add key-load-delay config for testing
      - trim connShutdown which is no longer needed
      - rioFdsetWrite -> rioFdWrite - simplified since there's no longer need to write to multiple FDs
      - don't detect rdb child exited (don't call wait3) until we detect the pipe is closed
      - Cleanup bad optimization from rio.c, add another one
      5a477946
    • Yossi Gottlieb's avatar
      TLS: Connections refactoring and TLS support. · b087dd1d
      Yossi Gottlieb authored
      * Introduce a connection abstraction layer for all socket operations and
      integrate it across the code base.
      * Provide an optional TLS connections implementation based on OpenSSL.
      * Pull a newer version of hiredis with TLS support.
      * Tests, redis-cli updates for TLS support.
      b087dd1d
  19. 27 Sep, 2019 2 commits
  20. 06 Sep, 2019 1 commit
  21. 05 Sep, 2019 1 commit
  22. 22 Jul, 2019 1 commit
  23. 19 Jul, 2019 1 commit
  24. 18 Jul, 2019 1 commit