1. 20 Jul, 2020 5 commits
    • Oran Agra's avatar
      stabilize tests that look for log lines (#7367) · 2b5f2319
      Oran Agra authored
      tests were sensitive to additional log lines appearing in the log
      causing the search to come empty handed.
      
      instead of just looking for the n last log lines, capture the log lines
      before performing the action, and then search from that offset.
      
      (cherry picked from commit 8e76e134)
      2b5f2319
    • Oran Agra's avatar
      tests/valgrind: don't use debug restart (#7404) · 1104113c
      Oran Agra authored
      * tests/valgrind: don't use debug restart
      
      DEBUG REATART causes two issues:
      1. it uses execve which replaces the original process and valgrind doesn't
         have a chance to check for errors, so leaks go unreported.
      2. valgrind report invalid calls to close() which we're unable to resolve.
      
      So now the tests use restart_server mechanism in the tests, that terminates
      the old server and starts a new one, new PID, but same stdout, stderr.
      
      since the stderr can contain two or more valgrind report, it is not enough
      to just check for the absence of leaks, we also need to check for some known
      errors, we do both, and fail if we either find an error, or can't find a
      report saying there are no leaks.
      
      other changes:
      - when killing a server that was already terminated we check for leaks too.
      - adding DEBUG LEAK which was used to test it.
      - adding --trace-children to valgrind, although no longer needed.
      - since the stdout contains two or more runs, we need slightly different way
        of checking if the new process is up (explicitly looking for the new PID)
      - move the code that handles --wait-server to happen earlier (before
        watching the startup message in the log), and serve the restarted server too.
      
      * squashme - CR fixes
      
      (cherry picked from commit 69ade873)
      1104113c
    • antirez's avatar
      LPOS: option FIRST renamed RANK. · 17aaf5ec
      antirez authored
      (cherry picked from commit a5a3a7bb)
      17aaf5ec
    • Oran Agra's avatar
      EXEC always fails with EXECABORT and multi-state is cleared · 05e483cb
      Oran Agra authored
      In order to support the use of multi-exec in pipeline, it is important that
      MULTI and EXEC are never rejected and it is easy for the client to know if the
      connection is still in multi state.
      
      It was easy to make sure MULTI and DISCARD never fail (done by previous
      commits) since these only change the client state and don't do any actual
      change in the server, but EXEC is a different story.
      
      Since in the past, it was possible for clients to handle some EXEC errors and
      retry the EXEC, we now can't affort to return any error on EXEC other than
      EXECABORT, which now carries with it the real reason for the abort too.
      
      Other fixes in this commit:
      - Some checks that where performed at the time of queuing need to be re-
        validated when EXEC runs, for instance if the transaction contains writes
        commands, it needs to be aborted. there was one check that was already done
        in execCommand (-READONLY), but other checks where missing: -OOM, -MISCONF,
        -NOREPLICAS, -MASTERDOWN
      - When a command is rejected by processCommand it was rejected with addReply,
        which was not recognized as an error in case the bad command came from the
        master. this will enable to count or MONITOR these errors in the future.
      - make it easier for tests to create additional (non deferred) clients.
      - add tests for the fixes of this commit.
      
      (cherry picked from commit 65a3307b)
      05e483cb
    • meir@redislabs.com's avatar
      Fix RM_ScanKey module api not to return int encoded strings · 51e17845
      meir@redislabs.com authored
      The scan key module API provides the scan callback with the current
      field name and value (if it exists). Those arguments are RedisModuleString*
      which means it supposes to point to robj which is encoded as a string.
      Using createStringObjectFromLongLong function might return robj that
      points to an integer and so break a module that tries for example to
      use RedisModule_StringPtrLen on the given field/value.
      
      The PR introduces a fix that uses the createObject function and sdsfromlonglong function.
      Using those function promise that the field and value pass to the to the
      scan callback will be Strings.
      
      The PR also changes the Scan test module to use RedisModule_StringPtrLen
      to catch the issue. without this, the issue is hidden because
      RedisModule_ReplyWithString knows to handle integer encoding of the
      given robj (RedisModuleString).
      
      The PR also introduces a new test to verify the issue is solved.
      
      (cherry picked from commit a89bf734)
      51e17845
  2. 12 Jun, 2020 1 commit
  3. 09 Jun, 2020 2 commits
  4. 06 Jun, 2020 1 commit
  5. 28 May, 2020 10 commits
  6. 22 May, 2020 7 commits
  7. 15 May, 2020 1 commit
    • Yossi Gottlieb's avatar
      TLS: Fix test failures on recent Debian/Ubuntu. · 16ba33c0
      Yossi Gottlieb authored
      Seems like on some systems choosing specific TLS v1/v1.1 versions no
      longer works as expected. Test is reduced for v1.2 now which is still
      good enough to test the mechansim, and matters most anyway.
      16ba33c0
  8. 14 May, 2020 3 commits
    • Oran Agra's avatar
      fix redis 6.0 not freeing closed connections during loading. · 9da134cd
      Oran Agra authored
      This bug was introduced by a recent change in which readQueryFromClient
      is using freeClientAsync, and despite the fact that now
      freeClientsInAsyncFreeQueue is in beforeSleep, that's not enough since
      it's not called during loading in processEventsWhileBlocked.
      furthermore, afterSleep was called in that case but beforeSleep wasn't.
      
      This bug also caused slowness sine the level-triggered mode of epoll
      kept signaling these connections as readable causing us to keep doing
      connRead again and again for ll of these, which keep accumulating.
      
      now both before and after sleep are called, but not all of their actions
      are performed during loading, some are only reserved for the main loop.
      
      fixes issue #7215
      9da134cd
    • antirez's avatar
      Regression test for #7249. · f7f219a1
      antirez authored
      f7f219a1
    • Oran Agra's avatar
      fix unstable replication test · 5c41802d
      Oran Agra authored
      this test which has coverage for varoius flows of diskless master was
      failing randomly from time to time.
      
      the failure was:
      [err]: diskless all replicas drop during rdb pipe in tests/integration/replication.tcl
      log message of '*Diskless rdb transfer, last replica dropped, killing fork child*' not found
      
      what seemed to have happened is that the master didn't detect that all
      replicas dropped by the time the replication ended, it thought that one
      replica is still connected.
      
      now the test takes a few seconds longer but it seems stable.
      5c41802d
  9. 08 May, 2020 3 commits
    • antirez's avatar
      Test: --dont-clean should do first cleanup. · e48c3731
      antirez authored
      e48c3731
    • zhenwei pi's avatar
      Support setcpuaffinity on linux/bsd · d6436eb7
      zhenwei pi authored
      Currently, there are several types of threads/child processes of a
      redis server. Sometimes we need deeply optimise the performance of
      redis, so we would like to isolate threads/processes.
      
      There were some discussion about cpu affinity cases in the issue:
      https://github.com/antirez/redis/issues/2863
      
      
      
      So implement cpu affinity setting by redis.conf in this patch, then
      we can config server_cpulist/bio_cpulist/aof_rewrite_cpulist/
      bgsave_cpulist by cpu list.
      
      Examples of cpulist in redis.conf:
      server_cpulist 0-7:2      means cpu affinity 0,2,4,6
      bio_cpulist 1,3           means cpu affinity 1,3
      aof_rewrite_cpulist 8-11  means cpu affinity 8,9,10,11
      bgsave_cpulist 1,10-11    means cpu affinity 1,10,11
      
      Test on linux/freebsd, both work fine.
      Signed-off-by: default avatarzhenwei pi <pizhenwei@bytedance.com>
      d6436eb7
    • Oran Agra's avatar
      add daily github actions with libc malloc and valgrind · 3d3861dd
      Oran Agra authored
      * fix memlry leaks with diskless replica short read.
      * fix a few timing issues with valgrind runs
      * fix issue with valgrind and watchdog schedule signal
      
      about the valgrind WD issue:
      the stack trace test in logging.tcl, has issues with valgrind:
      ==28808== Can't extend stack to 0x1ffeffdb38 during signal delivery for thread 1:
      ==28808==   too small or bad protection modes
      
      it seems to be some valgrind bug with SA_ONSTACK.
      SA_ONSTACK seems unneeded since WD is not recursive (SA_NODEFER was removed),
      also, not sure if it's even valid without a call to sigaltstack()
      3d3861dd
  10. 30 Apr, 2020 1 commit
  11. 28 Apr, 2020 1 commit
  12. 27 Apr, 2020 2 commits
    • Oran Agra's avatar
      Keep track of meaningful replication offset in replicas too · e4d2bb62
      Oran Agra authored
      Now both master and replicas keep track of the last replication offset
      that contains meaningful data (ignoring the tailing pings), and both
      trim that tail from the replication backlog, and the offset with which
      they try to use for psync.
      
      the implication is that if someone missed some pings, or even have
      excessive pings that the promoted replica has, it'll still be able to
      psync (avoid full sync).
      
      the downside (which was already committed) is that replicas running old
      code may fail to psync, since the promoted replica trims pings form it's
      backlog.
      
      This commit adds a test that reproduces several cases of promotions and
      demotions with stale and non-stale pings
      
      Background:
      The mearningful offset on the master was added recently to solve a problem were
      the master is left all alone, injecting PINGs into it's backlog when no one is
      listening and then gets demoted and tries to replicate from a replica that didn't
      have any of the PINGs (or at least not the last ones).
      
      however, consider this case:
      master A has two replicas (B and C) replicating directly from it.
      there's no traffic at all, and also no network issues, just many pings in the
      tail of the backlog. now B gets promoted, A becomes a replica of B, and C
      remains a replica of A. when A gets demoted, it trims the pings from its
      backlog, and successfully replicate from B. however, C is still aware of
      these PINGs, when it'll disconnect and re-connect to A, it'll ask for something
      that's not in the backlog anymore (since A trimmed the tail of it's backlog),
      and be forced to do a full sync (something it didn't have to do before the
      meaningful offset fix).
      
      Besides that, the psync2 test was always failing randomly here and there, it
      turns out the reason were PINGs. Investigating it shows the following scenario:
      
      cycle 1: redis #1 is master, and all the rest are direct replicas of #1
      cycle 2: redis #2 is promoted to master, #1 is a replica of #2 and #3 is replica of #1
      now we see that when #1 is demoted it prints:
      17339:S 21 Apr 2020 11:16:38.523 * Using the meaningful offset 3929963 instead of 3929977 to exclude the final PINGs (14 bytes difference)
      17339:S 21 Apr 2020 11:16:39.391 * Trying a partial resynchronization (request e2b3f8817735fdfe5fa4626766daa938b61419e5:3929964).
      17339:S 21 Apr 2020 11:16:39.392 * Successful partial resynchronization with master.
      and when #3 connects to the demoted #2, #2 says:
      17339:S 21 Apr 2020 11:16:40.084 * Partial resynchronization not accepted: Requested offset for secondary ID was 3929978, but I can reply up to 3929964
      
      so the issue here is that the meaningful offset feature saved the day for the
      demoted master (since it needs to sync from a replica that didn't get the last
      ping), but it didn't help one of the other replicas which did get the last ping.
      e4d2bb62
    • Guy Benoish's avatar
      Add the stream tag to XSETID tests · 43329c9b
      Guy Benoish authored
      43329c9b
  13. 24 Apr, 2020 3 commits
    • antirez's avatar
      LCS -> STRALGO LCS. · 3722f89f
      antirez authored
      STRALGO should be a container for mostly read-only string
      algorithms in Redis. The algorithms should have two main
      characteristics:
      
      1. They should be non trivial to compute, and often not part of
      programming language standard libraries.
      2. They should be fast enough that it is a good idea to have optimized C
      implementations.
      
      Next thing I would love to see? A small strings compression algorithm.
      3722f89f
    • yanhui13's avatar
      add tcl test for cluster slots · f03f1fad
      yanhui13 authored
      f03f1fad
    • antirez's avatar
      Tracking: test expired keys notifications. · 06917e58
      antirez authored
      06917e58