1. 27 Sep, 2020 2 commits
  2. 25 Sep, 2020 3 commits
    • Wang Yuan's avatar
      Set 'loading' and 'shutdown_asap' to volatile sig_atomic_t type (#7845) · f1863a1f
      Wang Yuan authored
      We may access and modify these two variables in signal handler function,
      to guarantee them async-signal-safe, so we should set them to volatile
      sig_atomic_t type.
      
      It doesn't look like this could have caused any real issue, and it seems that
      signals are handled in main thread on most platforms. But we want to follow C
      and POSIX standard in signal handler function.
      f1863a1f
    • Uri Shachar's avatar
      Fix config rewrite file handling to make it really atomic (#7824) · c30bd02c
      Uri Shachar authored
      Make sure we handle short writes correctly, sync to disk after writing  and use
      rename to make sure the replacement is actually atomic.
      In any case of failure old configuration will remain in place.
      
      Also, add some additional logging to make it easier to diagnose rewrite problems.
      c30bd02c
    • WuYunlong's avatar
      Add fsync to readSyncBulkPayload(). (#7839) · 0d62caab
      WuYunlong authored
      We should sync temp DB file before renaming as rdb_fsync_range does not use
      flag `SYNC_FILE_RANGE_WAIT_AFTER`.
      
      Refer to `Linux Programmer's Manual`:
      SYNC_FILE_RANGE_WAIT_AFTER
          Wait upon write-out of all pages in the range after performing any write.
      0d62caab
  3. 24 Sep, 2020 6 commits
    • Wen Hui's avatar
      rdb.c: handle fclose error case differently to avoid double fclose (#7307) · 323029ba
      Wen Hui authored
      When fclose would fail, the previous implementation would have attempted to do fclose again
      this can in theory lead to segfault.
      
      other changes:
      check for non-zero return value as failure rather than a specific error code.
      this doesn't fix a real bug, just a minor cleanup.
      323029ba
    • Wang Yuan's avatar
      Don't write replies if close the client ASAP (#7202) · 57709c4b
      Wang Yuan authored
      
      
      Before this commit, we would have continued to add replies to the reply buffer even if client
      output buffer limit is reached, so the used memory would keep increasing over the configured limit.
      What's more, we shouldn’t write any reply to the client if it is set 'CLIENT_CLOSE_ASAP' flag
      because that doesn't conform to its definition and we will close all clients flagged with
      'CLIENT_CLOSE_ASAP' in ‘beforeSleep’.
      
      Because of code execution order, before this, we may firstly write to part of the replies to
      the socket before disconnecting it, but in fact, we may can’t send the full replies to clients
      since OS socket buffer is limited. But this unexpected behavior makes some commands work well,
      for instance ACL DELUSER, if the client deletes the current user, we need to send reply to client
      and close the connection, but before, we close the client firstly and write the reply to reply
      buffer. secondly, we shouldn't do this despite the fact it works well in most cases.
      
      We add a flag 'CLIENT_CLOSE_AFTER_COMMAND' to mark clients, this flag means we will close the
      client after executing commands and send all entire replies, so that we can write replies to
      reply buffer during executing commands, send replies to clients, and close them later.
      
      We also fix some implicit problems. If client output buffer limit is enforced in 'multi/exec',
      all commands will be executed completely in redis and clients will not read any reply instead of
      partial replies. Even more, if the client executes 'ACL deluser' the using user in 'multi/exec',
      it will not read the replies after 'ACL deluser' just like before executing 'client kill' itself
      in 'multi/exec'.
      
      We added some tests for output buffer limit breach during multi-exec and using a pipeline of
      many small commands rather than one with big response.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      57709c4b
    • Guy Korland's avatar
      Fix RedisModule_HashGet examples (#6697) · b464afb9
      Guy Korland authored
      b464afb9
    • valentinogeron's avatar
      Stream: Inconsistency between master and replica some XREADGROUP case (#7526) · 795c454d
      valentinogeron authored
      XREADGROUP auto-creates the consumer inside the consumer group the
      first time it saw it.
      When XREADGROUP is being used with NOACK option, the message will not
      be added into the client's PEL and XGROUP SETID would be propagated.
      When the replica gets the XGROUP SETID it will only update the last delivered
      id of the group, but will not create the consumer.
      
      So, in this commit XGROUP CREATECONSUMER is being added.
      Command pattern: XGROUP CREATECONSUMER <key> <group> <consumer>.
      
      When NOACK option is being used, createconsumer command would be
      propagated as well.
      
      In case of AOFREWRITE, consumer with an empty PEL would be saved with
      XGROUP CREATECONSUMER whereas consumer with pending entries would be
      saved with XCLAIM
      795c454d
    • bodong.ybd's avatar
      Refactor multi-key command get keys proc · b7ce583a
      bodong.ybd authored
      b7ce583a
    • bodong.ybd's avatar
      Add ZINTER/ZUNION command · e08bf166
      bodong.ybd authored
      Syntax: ZINTER/ZUNION numkeys key [key ...] [WEIGHTS weight [weight ...]]
      [AGGREGATE SUM|MIN|MAX] [WITHSCORES]
      
      see #7624
      e08bf166
  4. 23 Sep, 2020 4 commits
  5. 22 Sep, 2020 6 commits
  6. 21 Sep, 2020 5 commits
  7. 20 Sep, 2020 3 commits
  8. 19 Sep, 2020 1 commit
  9. 17 Sep, 2020 2 commits
    • Wang Yuan's avatar
      Remove tmp rdb file in background thread (#7762) · b002d2b4
      Wang Yuan authored
      We're already using bg_unlink in several places to delete the rdb file in the background,
      and avoid paying the cost of the deletion from our main thread.
      This commit uses bg_unlink to remove the temporary rdb file in the background too.
      
      However, in case we delete that rdb file just before exiting, we don't actually wait for the
      background thread or the main thread to delete it, and just let the OS clean up after us.
      i.e. we open the file, unlink it and exit with the fd still open.
      
      Furthermore, rdbRemoveTempFile can be called from a thread and was using snprintf which is
      not async-signal-safe, we now use ll2string instead.
      b002d2b4
    • Wang Yuan's avatar
      Implement redisAtomic to replace _Atomic C11 builtin (#7707) · 445a4b66
      Wang Yuan authored
      Redis 6.0 introduces I/O threads, it is so cool and efficient, we use C11
      _Atomic to establish inter-thread synchronization without mutex. But the
      compiler that must supports C11 _Atomic can compile redis code, that brings a
      lot of inconvenience since some common platforms can't support by default such
      as CentOS7, so we want to implement redis atomic type to make it more portable.
      
      We have implemented our atomic variable for redis that only has 'relaxed'
      operations in src/atomicvar.h, so we implement some operations with
      'sequentially-consistent', just like the default behavior of C11 _Atomic that
      can establish inter-thread synchronization. And we replace all uses of C11
      _Atomic with redis atomic variable.
      
      Our implementation of redis atomic variable uses C11 _Atomic, __atomic or
      __sync macros if available, it supports most common platforms, and we will
      detect automatically which feature we use. In Makefile we use a dummy file to
      detect if the compiler supports C11 _Atomic. Now for gcc, we can compile redis
      code theoretically if your gcc version is not less than 4.1.2(starts to support
      __sync_xxx operations). Otherwise, we remove use mutex fallback to implement
      redis atomic variable for performance and test. You will get compiling errors
      if your compiler doesn't support all features of above.
      
      For cover redis atomic variable tests, we add other CI jobs that build redis on
      CentOS6 and CentOS7 and workflow daily jobs that run the tests on them.
      For them, we just install gcc by default in order to cover different compiler
      versions, gcc is 4.4.7 by default installation on CentOS6 and 4.8.5 on CentOS7.
      
      We restore the feature that we can test redis with Helgrind to find data race
      errors. But you need install Valgrind in the default path configuration firstly
      before running your tests, since we use macros in helgrind.h to tell Helgrind
      inter-thread happens-before relationship explicitly for avoiding false positives.
      Please open an issue on github if you find data race errors relate to this commit.
      
      Unrelated:
      - Fix redefinition of typedef 'RedisModuleUserChangedFunc'
        For some old version compilers, they will report errors or warnings, if we
        re-define function type.
      445a4b66
  10. 16 Sep, 2020 4 commits
    • Oran Agra's avatar
      Add printf attribute and fix warnings and a minor bug (#7803) · 092cfca5
      Oran Agra authored
      The fix in error handling of rdbGenericLoadStringObject is an actual bugfix
      092cfca5
    • WuYunlong's avatar
      f8660233
    • WuYunlong's avatar
      bio: fix doFastMemoryTest. · 8b70cb0e
      WuYunlong authored
      If one thread got SIGSEGV, function sigsegvHandler() would be triggered,
      it would call bioKillThreads(). But call pthread_cancel() to cancel itself
      would make it block. Also note that if SIGSEGV is caught by bio thread, it
      should kill the main thread in order to give a positive report.
      8b70cb0e
    • Jim Brunner's avatar
      Incremental eviction processing (#7653) · 810e28a3
      Jim Brunner authored
      Rather than blindly evicting until maxmemory limit is achieved, this
      update adds a time limit to eviction.  While over the maxmemory limit,
      eviction will process before each command AND as a timeProc when no
      commands are running.
      
      This will reduce the latency impact on many cases, especially pathological
      cases like massive used memory increase during dict rehashing.
      
      There is a risk that some other edge cases (like massive pipelined use
      of MGET) could cause Redis memory usage to keep growing despite the
      eviction attempts, so a new maxmemory-eviction-tenacity config is
      introduced to let users mitigate that.
      810e28a3
  11. 15 Sep, 2020 2 commits
  12. 13 Sep, 2020 2 commits