1. 27 Oct, 2020 5 commits
    • Wang Yuan's avatar
      Don't write replies if close the client ASAP (#7202) · 757ad7cd
      Wang Yuan authored
      
      
      Before this commit, we would have continued to add replies to the reply buffer even if client
      output buffer limit is reached, so the used memory would keep increasing over the configured limit.
      What's more, we shouldn’t write any reply to the client if it is set 'CLIENT_CLOSE_ASAP' flag
      because that doesn't conform to its definition and we will close all clients flagged with
      'CLIENT_CLOSE_ASAP' in ‘beforeSleep’.
      
      Because of code execution order, before this, we may firstly write to part of the replies to
      the socket before disconnecting it, but in fact, we may can’t send the full replies to clients
      since OS socket buffer is limited. But this unexpected behavior makes some commands work well,
      for instance ACL DELUSER, if the client deletes the current user, we need to send reply to client
      and close the connection, but before, we close the client firstly and write the reply to reply
      buffer. secondly, we shouldn't do this despite the fact it works well in most cases.
      
      We add a flag 'CLIENT_CLOSE_AFTER_COMMAND' to mark clients, this flag means we will close the
      client after executing commands and send all entire replies, so that we can write replies to
      reply buffer during executing commands, send replies to clients, and close them later.
      
      We also fix some implicit problems. If client output buffer limit is enforced in 'multi/exec',
      all commands will be executed completely in redis and clients will not read any reply instead of
      partial replies. Even more, if the client executes 'ACL deluser' the using user in 'multi/exec',
      it will not read the replies after 'ACL deluser' just like before executing 'client kill' itself
      in 'multi/exec'.
      
      We added some tests for output buffer limit breach during multi-exec and using a pipeline of
      many small commands rather than one with big response.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      (cherry picked from commit 57709c4b)
      757ad7cd
    • Wen Hui's avatar
      Add Swapdb Module Event (#7804) · 88441019
      Wen Hui authored
      
      (cherry picked from commit dfe9714c)
      88441019
    • Wang Yuan's avatar
      Remove tmp rdb file in background thread (#7762) · 0bdddd3c
      Wang Yuan authored
      We're already using bg_unlink in several places to delete the rdb file in the background,
      and avoid paying the cost of the deletion from our main thread.
      This commit uses bg_unlink to remove the temporary rdb file in the background too.
      
      However, in case we delete that rdb file just before exiting, we don't actually wait for the
      background thread or the main thread to delete it, and just let the OS clean up after us.
      i.e. we open the file, unlink it and exit with the fd still open.
      
      Furthermore, rdbRemoveTempFile can be called from a thread and was using snprintf which is
      not async-signal-safe, we now use ll2string instead.
      
      (cherry picked from commit b002d2b4)
      0bdddd3c
    • Oran Agra's avatar
      fix broken PEXPIREAT test (#7791) · 5106a144
      Oran Agra authored
      This test was nearly always failing on MacOS github actions.
      This is because of bugs in the test that caused it to nearly always run
      all 3 attempts and just look at the last one as the pass/fail creteria.
      
      i.e. the test was nearly always running all 3 attempts and still sometimes
      succeed. this is because the break condition was different than the test
      completion condition.
      
      The reason the test succeeded is because the break condition tested the
      results of all 3 tests (PSETEX/PEXPIRE/PEXPIREAT), but the success check
      at the end was only testing the result of PSETEX.
      
      The reason the PEXPIREAT test nearly always failed is because it was
      getting the current time wrong: getting the current second and loosing
      the sub-section time, so the only chance for it to succeed is if it run
      right when a certain second started.
      
      Because i now get the time from redis, adding another round trip, i
      added another 100ms to the PEXPIRE test to make it less fragile, and
      also added many more attempts.
      
      Adding many more attempts before failure to account for slow platforms,
      github actions and valgrind
      
      (cherry picked from commit ed9bfe22)
      5106a144
    • Oran Agra's avatar
      Squash merging 125 typo/grammar/comment/doc PRs (#7773) · 2ae7f491
      Oran Agra authored
      List of squashed commits or PRs
      ===============================
      
      commit 66801ea
      Author: hwware <wen.hui.ware@gmail.com>
      Date:   Mon Jan 13 00:54:31 2020 -0500
      
          typo fix in acl.c
      
      commit 46f55db
      Author: Itamar Haber <itamar@redislabs.com>
      Date:   Sun Sep 6 18:24:11 2020 +0300
      
          Updates a couple of comments
      
          Specifically:
      
          * RM_AutoMemory completed instead of pointing to docs
          * Updated link to custom type doc
      
      commit 61a2aa0
      Author: xindoo <xindoo@qq.com>
      Date:   Tue Sep 1 19:24:59 2020 +0800
      
          Correct errors in code comments
      
      commit a5871d1
      Author: yz1509 <pro-756@qq.com>
      Date:   Tue Sep 1 18:36:06 2020 +0800
      
          fix typos in module.c
      
      commit 41eede7
      Author: bookug <bookug@qq.com>
      Date:   Sat Aug 15 01:11:33 2020 +0800
      
          docs: fix typos in comments
      
      commit c303c84
      Author: lazy-snail <ws.niu@outlook.com>
      Date:   Fri Aug 7 11:15:44 2020 +0800
      
          fix spelling in redis.conf
      
      commit 1eb76bf
      Author: zhujian <zhujianxyz@gmail.com>...
      2ae7f491
  2. 10 Sep, 2020 6 commits
  3. 01 Sep, 2020 11 commits
  4. 20 Jul, 2020 9 commits
    • WuYunlong's avatar
      Fix command help for unexpected options (#7476) · b1a01fda
      WuYunlong authored
      
      (cherry picked from commit 93bdbf5a)
      b1a01fda
    • Oran Agra's avatar
      fix recently added time sensitive tests failing with valgrind (#7512) · aea4db2f
      Oran Agra authored
      interestingly the latency monitor test fails because valgrind is slow
      enough so that the time inside PEXPIREAT command from the moment of
      the first mstime() call to get the basetime until checkAlreadyExpired
      calls mstime() again is more than 1ms, and that test was too sensitive.
      
      using this opportunity to speed up the test (unrelated to the failure)
      the fix is just the longer time passed to PEXPIRE.
      
      (cherry picked from commit e5227aab)
      aea4db2f
    • Yossi Gottlieb's avatar
      TLS: Session caching configuration support. (#7420) · 7a536c29
      Yossi Gottlieb authored
      * TLS: Session caching configuration support.
      * TLS: Remove redundant config initialization.
      
      (cherry picked from commit 3e6f2b1a)
      7a536c29
    • Oran Agra's avatar
      RESTORE ABSTTL won't store expired keys into the db (#7472) · 95ba01b5
      Oran Agra authored
      Similarly to EXPIREAT with TTL in the past, which implicitly deletes the
      key and return success, RESTORE should not store key that are already
      expired into the db.
      When used together with REPLACE it should emit a DEL to keyspace
      notification and replication stream.
      
      (cherry picked from commit 5977a948)
      95ba01b5
    • Oran Agra's avatar
      skip a test that uses +inf on valgrind (#7440) · 33ca884c
      Oran Agra authored
      On some platforms strtold("+inf") with valgrind returns a non-inf result
      
      [err]: INCRBYFLOAT does not allow NaN or Infinity in tests/unit/type/incr.tcl
      Expected 'ERR*would produce*' to equal or match '1189731495357231765085759.....'
      
      (cherry picked from commit 909bc97c)
      33ca884c
    • Oran Agra's avatar
      stabilize tests that look for log lines (#7367) · 2b5f2319
      Oran Agra authored
      tests were sensitive to additional log lines appearing in the log
      causing the search to come empty handed.
      
      instead of just looking for the n last log lines, capture the log lines
      before performing the action, and then search from that offset.
      
      (cherry picked from commit 8e76e134)
      2b5f2319
    • antirez's avatar
      LPOS: option FIRST renamed RANK. · 17aaf5ec
      antirez authored
      (cherry picked from commit a5a3a7bb)
      17aaf5ec
    • Oran Agra's avatar
      EXEC always fails with EXECABORT and multi-state is cleared · 05e483cb
      Oran Agra authored
      In order to support the use of multi-exec in pipeline, it is important that
      MULTI and EXEC are never rejected and it is easy for the client to know if the
      connection is still in multi state.
      
      It was easy to make sure MULTI and DISCARD never fail (done by previous
      commits) since these only change the client state and don't do any actual
      change in the server, but EXEC is a different story.
      
      Since in the past, it was possible for clients to handle some EXEC errors and
      retry the EXEC, we now can't affort to return any error on EXEC other than
      EXECABORT, which now carries with it the real reason for the abort too.
      
      Other fixes in this commit:
      - Some checks that where performed at the time of queuing need to be re-
        validated when EXEC runs, for instance if the transaction contains writes
        commands, it needs to be aborted. there was one check that was already done
        in execCommand (-READONLY), but other checks where missing: -OOM, -MISCONF,
        -NOREPLICAS, -MASTERDOWN
      - When a command is rejected by processCommand it was rejected with addReply,
        which was not recognized as an error in case the bad command came from the
        master. this will enable to count or MONITOR these errors in the future.
      - make it easier for tests to create additional (non deferred) clients.
      - add tests for the fixes of this commit.
      
      (cherry picked from commit 65a3307b)
      05e483cb
    • meir@redislabs.com's avatar
      Fix RM_ScanKey module api not to return int encoded strings · 51e17845
      meir@redislabs.com authored
      The scan key module API provides the scan callback with the current
      field name and value (if it exists). Those arguments are RedisModuleString*
      which means it supposes to point to robj which is encoded as a string.
      Using createStringObjectFromLongLong function might return robj that
      points to an integer and so break a module that tries for example to
      use RedisModule_StringPtrLen on the given field/value.
      
      The PR introduces a fix that uses the createObject function and sdsfromlonglong function.
      Using those function promise that the field and value pass to the to the
      scan callback will be Strings.
      
      The PR also changes the Scan test module to use RedisModule_StringPtrLen
      to catch the issue. without this, the issue is hidden because
      RedisModule_ReplyWithString knows to handle integer encoding of the
      given robj (RedisModuleString).
      
      The PR also introduces a new test to verify the issue is solved.
      
      (cherry picked from commit a89bf734)
      51e17845
  5. 12 Jun, 2020 1 commit
  6. 09 Jun, 2020 2 commits
  7. 28 May, 2020 1 commit
    • Oran Agra's avatar
      tests: each test client work on a distinct port range · a2ae4635
      Oran Agra authored
      apparently when running tests in parallel (the default of --clients 16),
      there's a chance for two tests to use the same port.
      specifically, one test might shutdown a master and still have the
      replica up, and then another test will re-use the port number of master
      for another master, and then that replica will connect to the master of
      the other test.
      
      this can cause a master to count too many full syncs and fail a test if
      we run the tests with --single integration/psync2 --loop --stop
      
      see Probmem 2 in #7314
      a2ae4635
  8. 22 May, 2020 1 commit
    • Oran Agra's avatar
      fix a rare active defrag edge case bug leading to stagnation · 436be349
      Oran Agra authored
      There's a rare case which leads to stagnation in the defragger, causing
      it to keep scanning the keyspace and do nothing (not moving any
      allocation), this happens when all the allocator slabs of a certain bin
      have the same % utilization, but the slab from which new allocations are
      made have a lower utilization.
      
      this commit fixes it by removing the current slab from the overall
      average utilization of the bin, and also eliminate any precision loss in
      the utilization calculation and move the decision about the defrag to
      reside inside jemalloc.
      
      and also add a test that consistently reproduce this issue.
      436be349
  9. 15 May, 2020 1 commit
    • Yossi Gottlieb's avatar
      TLS: Fix test failures on recent Debian/Ubuntu. · 16ba33c0
      Yossi Gottlieb authored
      Seems like on some systems choosing specific TLS v1/v1.1 versions no
      longer works as expected. Test is reduced for v1.2 now which is still
      good enough to test the mechansim, and matters most anyway.
      16ba33c0
  10. 14 May, 2020 1 commit
  11. 08 May, 2020 1 commit
    • zhenwei pi's avatar
      Support setcpuaffinity on linux/bsd · d6436eb7
      zhenwei pi authored
      Currently, there are several types of threads/child processes of a
      redis server. Sometimes we need deeply optimise the performance of
      redis, so we would like to isolate threads/processes.
      
      There were some discussion about cpu affinity cases in the issue:
      https://github.com/antirez/redis/issues/2863
      
      
      
      So implement cpu affinity setting by redis.conf in this patch, then
      we can config server_cpulist/bio_cpulist/aof_rewrite_cpulist/
      bgsave_cpulist by cpu list.
      
      Examples of cpulist in redis.conf:
      server_cpulist 0-7:2      means cpu affinity 0,2,4,6
      bio_cpulist 1,3           means cpu affinity 1,3
      aof_rewrite_cpulist 8-11  means cpu affinity 8,9,10,11
      bgsave_cpulist 1,10-11    means cpu affinity 1,10,11
      
      Test on linux/freebsd, both work fine.
      Signed-off-by: default avatarzhenwei pi <pizhenwei@bytedance.com>
      d6436eb7
  12. 30 Apr, 2020 1 commit