1. 02 Dec, 2020 2 commits
    • Wang Yuan's avatar
      Backup keys to slots map and restore when fail to sync if diskless-load type... · b55a827e
      Wang Yuan authored
      
      Backup keys to slots map and restore when fail to sync if diskless-load type is swapdb in cluster mode (#8108)
      
      When replica diskless-load type is swapdb in cluster mode, we didn't backup
      keys to slots map, so we will lose keys to slots map if fail to sync.
      Now we backup keys to slots map at first, and restore it properly when fail.
      
      This commit includes a refactory/cleanup of the backups mechanism (moving it to db.c and re-structuring it a bit).
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      b55a827e
    • luhuachao's avatar
      Modify help msg PING_BULK to PING_MBULK in benchmark (#8109) · 7885faf1
      luhuachao authored
      As described in redis-benchamrk help message 'The test names are the same as the ones produced as output.', In redis-benchmark output, we can only see PING_BULK, but the cmd `redis-benchmark -t ping_bulk` is not supported. We  have to run it with ping_mbulk which is not user friendly.
      7885faf1
  2. 01 Dec, 2020 3 commits
    • Madelyn Olson's avatar
      Getset fix (#8118) · 69b7113b
      Madelyn Olson authored
      
      
      * Fixed SET GET executing on wrong type
      Co-authored-by: default avatarMadelyn Olson <madelyneolson@gmail.com>
      69b7113b
    • sundb's avatar
      Improve dbid range check for SELECT, MOVE, COPY (#8085) · 3ba2281f
      sundb authored
      SELECT used to read the index into a `long` variable, and then pass it to a function
      that takes an `int`, possibly causing an overflow before the range check.
      
      Now all these commands use better and cleaner range check, and that also results in
      a slight change of the error response in case of an invalid database index.
      
      SELECT:
      in the past it would have returned either `-ERR invalid DB index` (if not a number),
      or `-ERR DB index is out of range` (if not between 1..16 or alike).
      now it'll return either `-ERR value is out of range` (if not a number), or
      `-ERR value is out of range, value must between -2147483648 and 2147483647`
      (if not in the range for an int), or `-ERR DB index is out of range`
      (if not between 0..16 or alike)
      
      
      MOVE:
      in the past it would only fail with `-ERR index out of range` no matter the reason.
      now return the same errors as the new ones for SELECT mentioned above.
      (i.e. unlike for SELECT even for a value like 17 we changed the error message)
      
      COPY:
      doesn't really matter how it behaved in the past (new command), new behavior is
      like the above two.
      3ba2281f
    • Itamar Haber's avatar
      Adds pub/sub channel patterns to ACL (#7993) · c1b1e8c3
      Itamar Haber authored
      Fixes #7923.
      
      This PR appropriates the special `&` symbol (because `@` and `*` are taken),
      followed by a literal value or pattern for describing the Pub/Sub patterns that
      an ACL user can interact with. It is similar to the existing key patterns
      mechanism in function (additive) and implementation (copy-pasta). It also adds
      the allchannels and resetchannels ACL keywords, naturally.
      
      The default user is given allchannels permissions, whereas new users get
      whatever is defined by the acl-pubsub-default configuration directive. For
      backward compatibility in 6.2, the default of this directive is allchannels but
      this is likely to be changed to resetchannels in the next major version for
      stronger default security settings.
      
      Unless allchannels is set for the user, channel access permissions are checked
      as follows :
      * Calls to both PUBLISH and SUBSCRIBE will fail unless a pattern matching the
        argumentative channel name(s) exists for the user.
      * Calls to PSUBSCRIBE will fail unless the pattern(s) provided as an argument
        literally exist(s) in the user's list.
      
      Such failures are logged to the ACL log.
      
      Runtime changes to channel permissions for a user with existing subscribing
      clients cause said clients to disconnect unless the new permissions permit the
      connections to continue. Note, however, that PSUBSCRIBErs' patterns are matched
      literally, so given the change bar:* -> b*, pattern subscribers to bar:* will be
      disconnected.
      
      Notes/questions:
      * UNSUBSCRIBE, PUNSUBSCRIBE and PUBSUB remain unprotected due to lack of reasons
        for touching them.
      c1b1e8c3
  3. 30 Nov, 2020 2 commits
  4. 29 Nov, 2020 1 commit
    • guybe7's avatar
      XPENDING with IDLE (#7972) · ada2ac9a
      guybe7 authored
      Used to filter stream pending entries by their idle-time,
      useful for XCLAIMing entries that have not been processed
      for some time
      ada2ac9a
  5. 26 Nov, 2020 1 commit
  6. 25 Nov, 2020 4 commits
  7. 24 Nov, 2020 2 commits
  8. 23 Nov, 2020 1 commit
  9. 22 Nov, 2020 5 commits
    • xindoo's avatar
      0b1d89d7
    • Yossi Gottlieb's avatar
      Clean up building with USE_SYSTEMD. (#8073) · 08d3e929
      Yossi Gottlieb authored
      When USE_SYSTEMD=yes is specified, try to use pkg-config to determine
      libsystemd linker flags. If not found, silently fall back to simply
      using "-lsystemd".
      
      We now use a LIBSYSTEMD_LIBS variable so users can explicitly override
      it and specify their own library.
      
      If USE_SYSTEMD is unspecified the old behavior of auto-enabling it if
      both pkg-config and libsystemd are available is retained.
      08d3e929
    • Wang Yuan's avatar
      Fix diskless replication failure when has non-rdb child process (#8070) · f207e168
      Wang Yuan authored
      If we enable diskless replication, set repl-diskless-sync-delay to 0,
      and master has non-rdb child process such as rewrite aof child, master
      will try to start to a new BGSAVE but fails immediately (before fork)
      when replicas ask for full synchronization, and master always fails
      to start a new BGSAVE and disconnects with replicas until non-rdb
      child process exists.
      
      this bug was introduced in #6271 (not yet released in 6.0.x)
      f207e168
    • Oran Agra's avatar
      Fix bug with module GIL being released prematurely (#8061) · e6fa4738
      Oran Agra authored
      This is hopefully usually harmles.
      The server.ready_keys will usually be empty so the code after releasing
      the GIL will soon be done.
      The only case where it'll actually process things is when a module
      releases a client (or module) blocked on a key, by triggering this NOT
      from within a command (e.g. a timer event).
      
      This bug was introduced in redis 6.0.9, see #7903
      e6fa4738
    • Oran Agra's avatar
      Fix oom-score-adj-values range, abs options, and bug when used in config file (#8046) · 61954951
      Oran Agra authored
      Fix: When oom-score-adj-values is provided in the config file after
      oom-score-adj yes, it'll take an immediate action, before
      readOOMScoreAdj was acquired, resulting in an error (out of range score
      due to uninitialized value. delay the reaction the real call is made by
      main().
      
      Since the values are clamped to -1000..1000, and they're
      applied as an offset from the value at startup (which may be -1000), we
      need to allow the offsets to reach to +2000 so that a value of +1000 is
      achievable in case the value at startup was -1000.
      
      Adding an option for absolute values rather than relative ones.
      61954951
  10. 20 Nov, 2020 1 commit
  11. 18 Nov, 2020 1 commit
    • guybe7's avatar
      EXISTS should not alter LRU, OBJECT should not reveal expired keys on replica (#8016) · f8ae9917
      guybe7 authored
      The bug was introduced by #5021 which only attempted avoid EXIST on an
      already expired key from returning 1 on a replica.
      
      Before that commit, dbExists was used instead of
      lookupKeyRead (which had an undesired effect to "touch" the LRU/LFU)
      
      Other than that, this commit fixes OBJECT to also come empty handed on
      expired keys in replica.
      
      And DEBUG DIGEST-VALUE to behave like DEBUG OBJECT (get the data from
      the key regardless of it's expired state)
      f8ae9917
  12. 17 Nov, 2020 4 commits
    • Meir Shpilraien (Spielrein)'s avatar
      Unified MULTI, LUA, and RM_Call with respect to blocking commands (#8025) · d87a0d02
      Meir Shpilraien (Spielrein) authored
      
      
      Blocking command should not be used with MULTI, LUA, and RM_Call. This is because,
      the caller, who executes the command in this context, expects a reply.
      
      Today, LUA and MULTI have a special (and different) treatment to blocking commands:
      
      LUA   - Most commands are marked with no-script flag which are checked when executing
      and command from LUA, commands that are not marked (like XREAD) verify that their
      blocking mode is not used inside LUA (by checking the CLIENT_LUA client flag).
      MULTI - Command that is going to block, first verify that the client is not inside
      multi (by checking the CLIENT_MULTI client flag). If the client is inside multi, they
      return a result which is a match to the empty key with no timeout (for example blpop
      inside MULTI will act as lpop)
      For modules that perform RM_Call with blocking command, the returned results type is
      REDISMODULE_REPLY_UNKNOWN and the caller can not really know what happened.
      
      Disadvantages of the current state are:
      
      No unified approach, LUA, MULTI, and RM_Call, each has a different treatment
      Module can not safely execute blocking command (and get reply or error).
      Though It is true that modules are not like LUA or MULTI and should be smarter not
      to execute blocking commands on RM_Call, sometimes you want to execute a command base
      on client input (for example if you create a module that provides a new scripting
      language like javascript or python).
      While modules (on modules command) can check for REDISMODULE_CTX_FLAGS_LUA or
      REDISMODULE_CTX_FLAGS_MULTI to know not to block the client, there is no way to
      check if the command came from another module using RM_Call. So there is no way
      for a module to know not to block another module RM_Call execution.
      
      This commit adds a way to unify the treatment for blocking clients by introducing
      a new CLIENT_DENY_BLOCKING client flag. On LUA, MULTI, and RM_Call the new flag
      turned on to signify that the client should not be blocked. A blocking command
      verifies that the flag is turned off before blocking. If a blocking command sees
      that the CLIENT_DENY_BLOCKING flag is on, it's not blocking and return results
      which are matches to empty key with no timeout (as MULTI does today).
      
      The new flag is checked on the following commands:
      
      List blocking commands: BLPOP, BRPOP, BRPOPLPUSH, BLMOVE,
      Zset blocking commands: BZPOPMIN, BZPOPMAX
      Stream blocking commands: XREAD, XREADGROUP
      SUBSCRIBE, PSUBSCRIBE, MONITOR
      In addition, the new flag is turned on inside the AOF client, we do not want to
      block the AOF client to prevent deadlocks and commands ordering issues (and there
      is also an existing assert in the code that verifies it).
      
      To keep backward compatibility on LUA, all the no-script flags on existing commands
      were kept untouched. In addition, a LUA special treatment on XREAD and XREADGROUP was kept.
      
      To keep backward compatibility on MULTI (which today allows SUBSCRIBE, and PSUBSCRIBE).
      We added a special treatment on those commands to allow executing them on MULTI.
      
      The only backward compatibility issue that this PR introduces is that now MONITOR
      is not allowed inside MULTI.
      
      Tests were added to verify blocking commands are not blocking the client on LUA, MULTI,
      or RM_Call. Tests were added to verify the module can check for CLIENT_DENY_BLOCKING flag.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      Co-authored-by: default avatarItamar Haber <itamar@redislabs.com>
      d87a0d02
    • thomaston's avatar
      ZREVRANGEBYSCORE Optimization for out of range offset (#5773) · 39f716a1
      thomaston authored
      
      
      ZREVRANGEBYSCORE key max min [WITHSCORES] [LIMIT offset count]
      When the offset is too large, the query is very slow. Especially when the offset is greater than the length of zset it is easy to determine whether the offset is greater than the length of zset at first, and If it exceed the length of zset, then return directly.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      39f716a1
    • Yossi Gottlieb's avatar
      Improve and clean up supervised process support. (#8036) · d638b058
      Yossi Gottlieb authored
      * Configuration file default should now be "auto".
      * Expose "process_supervised" as an info field.
      * Log messages improvements (clarify required systemd config, report
        auto-detected supervision mode, etc.)
      * Set server.supervised properly, so it can take precedence of
        "daemonize" configuration.
      * Produce clear warning if systemd is detected/requested but executable
        is compiled without support for it, instead of silently ignoring.
      * Handle systemd notification error on startup, and turn off supervised
        mode if it failed.
      d638b058
    • swamp0407's avatar
      Add COPY command (#7953) · ea7cf737
      swamp0407 authored
      
      
      Syntax:
      COPY <key> <new-key> [DB <dest-db>] [REPLACE]
      
      No support for module keys yet.
      
      Co-authored-by: tmgauss
      Co-authored-by: default avatarItamar Haber <itamar@redislabs.com>
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      ea7cf737
  13. 16 Nov, 2020 2 commits
    • Oran Agra's avatar
      Fix memory leaks in newly added ZDIFF (#8056) · 9812e889
      Oran Agra authored
      9812e889
    • chenyangyang's avatar
      Modules callbacks for lazy free effort, and unlink (#7912) · c1aaad06
      chenyangyang authored
      Add two optional callbacks to the RedisModuleTypeMethods structure, which is `free_effort`
      and `unlink`. the `free_effort` callback indicates the effort required to free a module memory.
      Currently, if the effort exceeds LAZYFREE_THRESHOLD, the module memory may be released
      asynchronously. the `unlink` callback indicates the key has been removed from the DB by redis, and
      may soon be freed by a background thread.
      
      Add `lazyfreed_objects` info field, which represents the number of objects that have been
      lazyfreed since redis was started.
      
      Add `RM_GetTypeMethodVersion` API, which return the current redis-server runtime value of
      `REDISMODULE_TYPE_METHOD_VERSION`. You can use that when calling `RM_CreateDataType` to know
      which fields of RedisModuleTypeMethods are gonna be supported and which will be ignored.
      c1aaad06
  14. 15 Nov, 2020 1 commit
  15. 13 Nov, 2020 1 commit
  16. 12 Nov, 2020 2 commits
    • Itamar Haber's avatar
      Adds user parsing to redis-cli URIs (#8048) · 92ec5925
      Itamar Haber authored
      92ec5925
    • Yash Ladha's avatar
      cleanup: move list pop logic to single function (#7997) · c170365d
      Yash Ladha authored
      BLPOP when there are elements in the list works in the same way as LPOP
      does. Due to this they also does the same repetitive action and logic
      for the same is written at two different places. This is a bad code
      practice as the one needs the context to change the BLPOP list pop code
      as well when the LPOP code gets changed.
      
      Separated the generic logic from LPOP to a function that is being used
      by the BLPOP code as well.
      c170365d
  17. 11 Nov, 2020 3 commits
  18. 10 Nov, 2020 3 commits
  19. 09 Nov, 2020 1 commit