1. 08 Aug, 2020 1 commit
    • Wang Yuan's avatar
      Fix applying zero offset to null pointer when creating moduleFreeContextReusedClient (#7323) · 1ef014ee
      Wang Yuan authored
      Before this fix we where attempting to select a db before creating db the DB, see: #7323
      
      This issue doesn't seem to have any implications, since the selected DB index is 0,
      the db pointer remains NULL, and will later be correctly set before using this dummy
      client for the first time.
      
      As we know, we call 'moduleInitModulesSystem()' before 'initServer()'. We will allocate
      memory for server.db in 'initServer', but we call 'createClient()' that will call 'selectDb()'
      in 'moduleInitModulesSystem()', before the databases where created. Instead, we should call
      'createClient()' for moduleFreeContextReusedClient after 'initServer()'.
      1ef014ee
  2. 07 Aug, 2020 2 commits
  3. 06 Aug, 2020 3 commits
    • Oran Agra's avatar
      Accelerate diskless master connections, and general re-connections (#6271) · c17e597d
      Oran Agra authored
      Diskless master has some inherent latencies.
      1) fork starts with delay from cron rather than immediately
      2) replica is put online only after an ACK. but the ACK
         was sent only once a second.
      3) but even if it would arrive immediately, it will not
         register in case cron didn't yet detect that the fork is done.
      
      Besides that, when a replica disconnects, it doesn't immediately
      attempts to re-connect, it waits for replication cron (one per second).
      in case it was already online, it may be important to try to re-connect
      as soon as possible, so that the backlog at the master doesn't vanish.
      
      In case it disconnected during rdb transfer, one can argue that it's
      not very important to re-connect immediately, but this is needed for the
      "diskless loading short read" test to be able to run 100 iterations in 5
      seconds, rather than 3 (waiting for replication cron re-connection)
      
      changes in this commit:
      1) sync command starts a fork immediately if no sync_delay is configured
      2) replica sends REPLCONF ACK when done reading the rdb (rather than on 1s cron)
      3) when a replica unexpectedly disconnets, it immediately tries to
         re-connect rather than waiting 1s
      4) when when a child exits, if there is another replica waiting, we spawn a new
         one right away, instead of waiting for 1s replicationCron.
      5) added a call to connectWithMaster from replicationSetMaster. which is called
         from the REPLICAOF command but also in 3 places in cluster.c, in all of
         these the connection attempt will now be immediate instead of delayed by 1
         second.
      
      side note:
      we can add a call to rdbPipeReadHandler in replconfCommand when getting
      a REPLCONF ACK from the replica to solve a race where the replica got
      the entire rdb and EOF marker before we detected that the pipe was
      closed.
      in the test i did see this race happens in one about of some 300 runs,
      but i concluded that this race is unlikely in real life (where the
      replica is on another host and we're more likely to first detect the
      pipe was closed.
      the test runs 100 iterations in 3 seconds, so in some cases it'll take 4
      seconds instead (waiting for another REPLCONF ACK).
      
      Removing unneeded startBgsaveForReplication from updateSlavesWaitingForBgsave
      Now that CheckChildrenDone is calling the new replicationStartPendingFork
      (extracted from serverCron) there's actually no need to call
      startBgsaveForReplication from updateSlavesWaitingForBgsave anymore,
      since as soon as updateSlavesWaitingForBgsave returns, CheckChildrenDone is
      calling replicationStartPendingFork that handles that anyway.
      The code in updateSlavesWaitingForBgsave had a bug in which it ignored
      repl-diskless-sync-delay, but removing that code shows that this bug was
      hiding another bug, which is that the max_idle should have used >= and
      not >, this one second delay has a big impact on my new test.
      c17e597d
    • Oran Agra's avatar
      Fix potential race in bugReportStart · 81f8524a
      Oran Agra authored
      this race would only happen when two threads paniced at the same time,
      and even then the only consequence is some extra log lines.
      
      race reported in #7391
      81f8524a
    • Oran Agra's avatar
      Assertion and panic, print crash log without generating SIGSEGV · 90b717e7
      Oran Agra authored
      This makes it possible to add tests that generate assertions, and run
      them with valgrind, making sure that there are no memory violations
      prior to the assertion.
      
      New config options:
      - crash-log-enabled - can be disabled for cleaner core dumps
      - crash-memcheck-enabled - useful for faster termination after a crash
      - use-exit-on-panic - to be used by the test suite so that valgrind can
        detect leaks and memory corruptions
      
      Other changes:
      - Crash log is printed even on system that dont HAVE_BACKTRACE, i.e. in
        both SIGSEGV and assert / panic
      - Assertion and panic won't print registers and code around EIP (which
        was useless), but will do fast memory test (which may still indicate
        that the assertion was due to memory corrpution)
      
      I had to reshuffle code in order to re-use it, so i extracted come code
      into function without actually doing any changes to the code:
      - logServerInfo
      - logModulesInfo
      - doFastMemoryTest (with the exception of it being conditional)
      - dumpCodeAroundEIP
      
      changes to the crash report on segfault:
      - logRegisters is called right after the stack trace (before info) done
        just in order to have more re-usable code
      - stack trace skips the first two items on the stack (the crash log and
        signal handler functions)
      90b717e7
  4. 05 Aug, 2020 2 commits
  5. 04 Aug, 2020 1 commit
    • Tyson Andre's avatar
      Add a ZMSCORE command returning an array of scores. (#7593) · f11f26cc
      Tyson Andre authored
      
      
      Syntax: `ZMSCORE KEY MEMBER [MEMBER ...]`
      
      This is an extension of #2359
      amended by Tyson Andre to work with the changed unstable API,
      add more tests, and consistently return an array.
      
      - It seemed as if it would be more likely to get reviewed
        after updating the implementation.
      
      Currently, multi commands or lua scripting to call zscore multiple times
      would almost definitely be less efficient than a native ZMSCORE
      for the following reasons:
      
      - Need to fetch the set from the string every time instead of reusing the C
        pointer.
      - Using pipelining or multi-commands would result in more bytes sent by
        the client for the repeated `ZMSCORE KEY` sections.
      - Need to specially encode the data and decode it from the client
        for lua-based solutions.
      - The fastest solution I've seen for large sets(thousands or millions)
        involves lua and a variadic ZADD, then a ZINTERSECT, then a ZRANGE 0 -1,
        then UNLINK of a temporary set (or lua). This is still inefficient.
      Co-authored-by: default avatarTyson Andre <tysonandre775@hotmail.com>
      f11f26cc
  6. 02 Aug, 2020 1 commit
  7. 31 Jul, 2020 2 commits
  8. 30 Jul, 2020 2 commits
  9. 29 Jul, 2020 4 commits
  10. 28 Jul, 2020 3 commits
  11. 27 Jul, 2020 1 commit
    • Yossi Gottlieb's avatar
      TLS: support cluster/replication without tls-port. · c75512d8
      Yossi Gottlieb authored
      Initialize and configure OpenSSL even when tls-port is not used, because
      we may still have tls-cluster or tls-replication.
      
      Also, make sure to reconfigure OpenSSL when these parameters are changed
      as TLS could have been enabled for the first time.
      c75512d8
  12. 26 Jul, 2020 1 commit
  13. 24 Jul, 2020 1 commit
  14. 23 Jul, 2020 2 commits
    • Meir Shpilraien (Spielrein)'s avatar
    • Oran Agra's avatar
      Fix harmless bug in rioConnRead (#7557) · 40d7fca3
      Oran Agra authored
      this code is in use only if the master is disk-based, and the replica is
      diskless. In this case we use a buffered reader, but we must avoid reading
      past the rdb file, into the command stream. which Luckly rdb.c doesn't
      really attempt to do (it knows how much it should read).
      
      When rioConnRead detects that the extra buffering attempt reaches beyond
      the read limit it should read less, but if the caller actually requested
      more, then it should return with an error rather than a short read. the
      bug would have resulted in short read.
      
      in order to fix it, the code must consider the real requested size, and
      not the extra buffering size.
      40d7fca3
  15. 22 Jul, 2020 1 commit
  16. 21 Jul, 2020 3 commits
    • WuYunlong's avatar
      Clarification on the bug that was fixed in PR #7539. (#7541) · f7f77a74
      WuYunlong authored
      Before that PR, processCommand() did not notice that cmd could be a module
      command in which case getkeys_proc member has a different meaning.
      
      The outcome was that a module command which doesn't take any key names in its
      arguments (similar to SLOWLOG) would be handled as if it might have key name arguments
      (similar to MEMORY), would consider cluster redirect but will end up with 0 keys
      after an excessive call to getKeysFromCommand, and eventually do the right thing.
      f7f77a74
    • Wen Hui's avatar
      Add missing calls to raxStop (#7532) · 4e8f2d68
      Wen Hui authored
      Since the dynamic allocations in raxIterator are only used for deep walks, memory
      leak due to missing call to raxStop can only happen for rax with key names longer
      than 32 bytes.
      
      Out of all the missing calls, the only ones that may lead to a leak are the rax
      for consumer groups and consumers, and these were only in AOFRW and rdbSave, which
      normally only happen in fork or at shutdown.
      4e8f2d68
    • Wen Hui's avatar
      2fbd0271
  17. 20 Jul, 2020 3 commits
    • zhaozhao.zz's avatar
    • WuYunlong's avatar
      Fix cluster redirect for module command with no firstkey. (#7539) · e4d7de60
      WuYunlong authored
      Before this commit, processCommand() did not notice that cmd could be a module command
      which declared `getkeys-api` and handled it for the purpose of cluster redirect it
      as if it doesn't use any keys.
      
      This commit fixed it by reusing the codes in addReplyCommand().
      e4d7de60
    • WuYunlong's avatar
      Refactor streamAppendItem() by deleting redundancy condition. (#7487) · 86fed3fe
      WuYunlong authored
      It will never happen that "lp != NULL && lp_bytes >= server.stream_node_max_bytes".
      Assume that "lp != NULL && lp_bytes >= server.stream_node_max_bytes",
      we got the following conditions:
      a. lp != NULL
      b. lp_bytes >= server.stream_node_max_bytes
      
      If server.stream_node_max_bytes is 0, given condition a, condition b is always satisfied
      If server.stream_node_max_bytes is not 0, given condition a and condition b, the codes just a
      	few lines above set lp to NULL, a controdiction with condition a
      
      So that condition b is recundant. We could delete it safely.
      86fed3fe
  18. 16 Jul, 2020 2 commits
  19. 15 Jul, 2020 5 commits