1. 27 Oct, 2020 32 commits
    • Wang Yuan's avatar
      Don't support Gopher if enable io threads to read queries (#7851) · 94180837
      Wang Yuan authored
      There's currently an issue with IO threads and gopher (issuing lookupKey from within the thread).
      simply fix is to just not support it for now.
      
      (cherry picked from commit c9f00bcc)
      94180837
    • Wang Yuan's avatar
      Set 'loading' and 'shutdown_asap' to volatile sig_atomic_t type (#7845) · 5d1ac2df
      Wang Yuan authored
      We may access and modify these two variables in signal handler function,
      to guarantee them async-signal-safe, so we should set them to volatile
      sig_atomic_t type.
      
      It doesn't look like this could have caused any real issue, and it seems that
      signals are handled in main thread on most platforms. But we want to follow C
      and POSIX standard in signal handler function.
      
      (cherry picked from commit f1863a1f)
      5d1ac2df
    • Uri Shachar's avatar
      Fix config rewrite file handling to make it really atomic (#7824) · 90555566
      Uri Shachar authored
      Make sure we handle short writes correctly, sync to disk after writing  and use
      rename to make sure the replacement is actually atomic.
      In any case of failure old configuration will remain in place.
      
      Also, add some additional logging to make it easier to diagnose rewrite problems.
      
      (cherry picked from commit c30bd02c)
      90555566
    • WuYunlong's avatar
      Add fsync to readSyncBulkPayload(). (#7839) · bb7e00b6
      WuYunlong authored
      We should sync temp DB file before renaming as rdb_fsync_range does not use
      flag `SYNC_FILE_RANGE_WAIT_AFTER`.
      
      Refer to `Linux Programmer's Manual`:
      SYNC_FILE_RANGE_WAIT_AFTER
          Wait upon write-out of all pages in the range after performing any write.
      
      (cherry picked from commit 0d62caab)
      bb7e00b6
    • Wen Hui's avatar
      rdb.c: handle fclose error case differently to avoid double fclose (#7307) · faa431c9
      Wen Hui authored
      When fclose would fail, the previous implementation would have attempted to do fclose again
      this can in theory lead to segfault.
      
      other changes:
      check for non-zero return value as failure rather than a specific error code.
      this doesn't fix a real bug, just a minor cleanup.
      
      (cherry picked from commit 323029ba)
      faa431c9
    • Wang Yuan's avatar
      Don't write replies if close the client ASAP (#7202) · 757ad7cd
      Wang Yuan authored
      
      
      Before this commit, we would have continued to add replies to the reply buffer even if client
      output buffer limit is reached, so the used memory would keep increasing over the configured limit.
      What's more, we shouldn’t write any reply to the client if it is set 'CLIENT_CLOSE_ASAP' flag
      because that doesn't conform to its definition and we will close all clients flagged with
      'CLIENT_CLOSE_ASAP' in ‘beforeSleep’.
      
      Because of code execution order, before this, we may firstly write to part of the replies to
      the socket before disconnecting it, but in fact, we may can’t send the full replies to clients
      since OS socket buffer is limited. But this unexpected behavior makes some commands work well,
      for instance ACL DELUSER, if the client deletes the current user, we need to send reply to client
      and close the connection, but before, we close the client firstly and write the reply to reply
      buffer. secondly, we shouldn't do this despite the fact it works well in most cases.
      
      We add a flag 'CLIENT_CLOSE_AFTER_COMMAND' to mark clients, this flag means we will close the
      client after executing commands and send all entire replies, so that we can write replies to
      reply buffer during executing commands, send replies to clients, and close them later.
      
      We also fix some implicit problems. If client output buffer limit is enforced in 'multi/exec',
      all commands will be executed completely in redis and clients will not read any reply instead of
      partial replies. Even more, if the client executes 'ACL deluser' the using user in 'multi/exec',
      it will not read the replies after 'ACL deluser' just like before executing 'client kill' itself
      in 'multi/exec'.
      
      We added some tests for output buffer limit breach during multi-exec and using a pipeline of
      many small commands rather than one with big response.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      (cherry picked from commit 57709c4b)
      757ad7cd
    • Guy Korland's avatar
      Fix RedisModule_HashGet examples (#6697) · 31491202
      Guy Korland authored
      
      (cherry picked from commit b464afb9)
      31491202
    • Oran Agra's avatar
      fix recently broken TLS build error, and add coverage for CI (#7833) · 26c1c601
      Oran Agra authored
      
      (cherry picked from commit a735bf5c)
      26c1c601
    • David CARLIER's avatar
      Further NetBSD update and build fixes. (#7831) · 2ccbba65
      David CARLIER authored
      mainly backtrace and register dump support.
      
      (cherry picked from commit c3edaa79)
      2ccbba65
    • WuYunlong's avatar
      Fix redundancy use of semicolon in do-while macros in ziplist.c. (#7832) · c73406c2
      WuYunlong authored
      this is very dangerous bug, but it looks like it didn't cause any harm.
      
      (cherry picked from commit 63cd4d4e)
      c73406c2
    • yixiang's avatar
      Fix connGetSocketError usage (#7811) · c1423c01
      yixiang authored
      
      (cherry picked from commit b96c3595)
      c1423c01
    • Oran Agra's avatar
      RM_GetContextFlags - document missing flags (#7821) · ecf51b0b
      Oran Agra authored
      
      (cherry picked from commit 26ca04a8)
      ecf51b0b
    • Yossi Gottlieb's avatar
      Fix occasional hangs on replication reconnection. (#7830) · 9d0388a0
      Yossi Gottlieb authored
      This happens only on diskless replicas when attempting to reconnect after 
      failing to load an RDB file. It is more likely to occur with larger datasets.
      
      After reconnection is initiated, replicationEmptyDbCallback() may get called 
      and try to write to an unconnected socket. This triggered another issue where
      the connection is put into an error state and the connect handler never gets
      called. The problem is a regression introduced by commit c17e597d.
      
      (cherry picked from commit 1980f639)
      9d0388a0
    • Ariel Shtul's avatar
      Fix redis-check-rdb support for modules aux data (#7826) · 6a4da495
      Ariel Shtul authored
      
      
      redis-check-rdb was unable to parse rdb files containing module aux data.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      (cherry picked from commit 63a05dde)
      6a4da495
    • Wen Hui's avatar
      refactor rewriteStreamObject code for adding missing streamIteratorStop call (#7829) · e8f772ef
      Wen Hui authored
      This commit adds streamIteratorStop call in rewriteStreamObject function in some of the return statement. Although currently this will not cause memory leak since stream id is only 16 bytes long.
      
      
      (cherry picked from commit 23b50bcc)
      e8f772ef
    • WuYunlong's avatar
      Make IO threads killable so that they can be canceled at any time. · 59866fcd
      WuYunlong authored
      This commit can be cherry picked to 6.0 only if we also cherry pick f8660233.
      
      (cherry picked from commit c37acb82)
      59866fcd
    • WuYunlong's avatar
      Make main thread killable so that it can be canceled at any time. · 4832cf4f
      WuYunlong authored
      Refine comment of makeThreadKillable().
      
      This commit can be backported to 5.0, only if we also backport 8b70cb0e
      
      .
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      (cherry picked from commit 647cac5b)
      4832cf4f
    • Oran Agra's avatar
      RM_GetContextFlags provides indication that we're in a fork child (#7783) · fd610ae0
      Oran Agra authored
      
      (cherry picked from commit 2458e548)
      fd610ae0
    • Wen Hui's avatar
      Add Swapdb Module Event (#7804) · 88441019
      Wen Hui authored
      
      (cherry picked from commit dfe9714c)
      88441019
    • Daniel Dai's avatar
      fix make warnings in debug.c MacOS (#7805) · aa1feec7
      Daniel Dai authored
      
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      (cherry picked from commit 1b3b7520)
      aa1feec7
    • David CARLIER's avatar
      debug.c: NetBSD build warning fix. (#7810) · e5e353c8
      David CARLIER authored
      The symbol base address is a const on this system.
      
      (cherry picked from commit eabe3eae)
      e5e353c8
    • Wang Yuan's avatar
      Remove tmp rdb file in background thread (#7762) · 0bdddd3c
      Wang Yuan authored
      We're already using bg_unlink in several places to delete the rdb file in the background,
      and avoid paying the cost of the deletion from our main thread.
      This commit uses bg_unlink to remove the temporary rdb file in the background too.
      
      However, in case we delete that rdb file just before exiting, we don't actually wait for the
      background thread or the main thread to delete it, and just let the OS clean up after us.
      i.e. we open the file, unlink it and exit with the fd still open.
      
      Furthermore, rdbRemoveTempFile can be called from a thread and was using snprintf which is
      not async-signal-safe, we now use ll2string instead.
      
      (cherry picked from commit b002d2b4)
      0bdddd3c
    • Oran Agra's avatar
      Add printf attribute and fix warnings and a minor bug (#7803) · 7bc07260
      Oran Agra authored
      The fix in error handling of rdbGenericLoadStringObject is an actual bugfix
      
      (cherry picked from commit 092cfca5)
      7bc07260
    • WuYunlong's avatar
      bio: doFastMemoryTest should try to kill io threads as well. · ea4bf91a
      WuYunlong authored
      (cherry picked from commit f8660233)
      ea4bf91a
    • WuYunlong's avatar
      bio: fix doFastMemoryTest. · 5d933226
      WuYunlong authored
      If one thread got SIGSEGV, function sigsegvHandler() would be triggered,
      it would call bioKillThreads(). But call pthread_cancel() to cancel itself
      would make it block. Also note that if SIGSEGV is caught by bio thread, it
      should kill the main thread in order to give a positive report.
      
      (cherry picked from commit 8b70cb0e)
      5d933226
    • Wen Hui's avatar
      correct OBJECT ENCODING response for stream type (#7797) · 7601ed34
      Wen Hui authored
      This commit makes stream object returning "stream" as encoding type in OBJECT ENCODING subcommand and DEBUG OBJECT command.
      
      Till now, it would return "unknown"
      
      (cherry picked from commit 6ff741b5)
      7601ed34
    • WuYunlong's avatar
      Clarify help text of tcl scripts. (#7798) · 405f041e
      WuYunlong authored
      Before this commit, following command did not show --tls option:
      ./runtest-cluster --help
      ./runtest-sentinel --help
      
      (cherry picked from commit 98c8ac0d)
      405f041e
    • Mykhailo Pylyp's avatar
      aa6b869c
    • Oran Agra's avatar
      Fix failing valgrind installation in github actions (#7792) · 00a767db
      Oran Agra authored
      These tests started failing every day on http 404 (not being able to
      install valgrind)
      
      (cherry picked from commit 78a6e5eb)
      00a767db
    • Oran Agra's avatar
      fix broken PEXPIREAT test (#7791) · 5106a144
      Oran Agra authored
      This test was nearly always failing on MacOS github actions.
      This is because of bugs in the test that caused it to nearly always run
      all 3 attempts and just look at the last one as the pass/fail creteria.
      
      i.e. the test was nearly always running all 3 attempts and still sometimes
      succeed. this is because the break condition was different than the test
      completion condition.
      
      The reason the test succeeded is because the break condition tested the
      results of all 3 tests (PSETEX/PEXPIRE/PEXPIREAT), but the success check
      at the end was only testing the result of PSETEX.
      
      The reason the PEXPIREAT test nearly always failed is because it was
      getting the current time wrong: getting the current second and loosing
      the sub-section time, so the only chance for it to succeed is if it run
      right when a certain second started.
      
      Because i now get the time from redis, adding another round trip, i
      added another 100ms to the PEXPIRE test to make it less fragile, and
      also added many more attempts.
      
      Adding many more attempts before failure to account for slow platforms,
      github actions and valgrind
      
      (cherry picked from commit ed9bfe22)
      5106a144
    • Wang Yuan's avatar
      Remove dead global variable 'lru_clock' (#7782) · 3efdbbc7
      Wang Yuan authored
      
      (cherry picked from commit 86511bbb)
      3efdbbc7
    • Oran Agra's avatar
      Squash merging 125 typo/grammar/comment/doc PRs (#7773) · 2ae7f491
      Oran Agra authored
      List of squashed commits or PRs
      ===============================
      
      commit 66801ea
      Author: hwware <wen.hui.ware@gmail.com>
      Date:   Mon Jan 13 00:54:31 2020 -0500
      
          typo fix in acl.c
      
      commit 46f55db
      Author: Itamar Haber <itamar@redislabs.com>
      Date:   Sun Sep 6 18:24:11 2020 +0300
      
          Updates a couple of comments
      
          Specifically:
      
          * RM_AutoMemory completed instead of pointing to docs
          * Updated link to custom type doc
      
      commit 61a2aa0
      Author: xindoo <xindoo@qq.com>
      Date:   Tue Sep 1 19:24:59 2020 +0800
      
          Correct errors in code comments
      
      commit a5871d1
      Author: yz1509 <pro-756@qq.com>
      Date:   Tue Sep 1 18:36:06 2020 +0800
      
          fix typos in module.c
      
      commit 41eede7
      Author: bookug <bookug@qq.com>
      Date:   Sat Aug 15 01:11:33 2020 +0800
      
          docs: fix typos in comments
      
      commit c303c84
      Author: lazy-snail <ws.niu@outlook.com>
      Date:   Fri Aug 7 11:15:44 2020 +0800
      
          fix spelling in redis.conf
      
      commit 1eb76bf
      Author: zhujian <zhujianxyz@gmail.com>...
      2ae7f491
  2. 10 Sep, 2020 8 commits