1. 08 Feb, 2021 5 commits
  2. 07 Feb, 2021 5 commits
  3. 05 Feb, 2021 3 commits
  4. 04 Feb, 2021 3 commits
  5. 03 Feb, 2021 1 commit
    • Yossi Gottlieb's avatar
      Fix FreeBSD tests and CI Daily issues. (#8438) · de6f3ad0
      Yossi Gottlieb authored
      * Add bash temporarily to allow sentinel fd leaks test to run.
      * Use vmactions-freebsd rdist sync to work around bind permission denied
        and slow execution issues.
      * Upgrade to tcl8.6 to be aligned with latest Ubuntu envs.
      * Concat all command executions to avoid ignoring failures.
      * Skip intensive fuzzer on FreeBSD. For some yet unknown reason, generate_fuzzy_traffic_on_key causes TCL to significantly bloat on FreeBSD resulting with out of memory.
      de6f3ad0
  6. 02 Feb, 2021 3 commits
    • Jonah H. Harris's avatar
      Optimizing sorted GEORADIUS COUNT with partial sorting. (#8326) · a3718cde
      Jonah H. Harris authored
      This commit provides an optimization, in terms of time, for all GEORADIUS*
      and GEOSEARCH* searches which utilize the default, sorted, COUNT clause.
      This is commonly used for nearest-neighbor (top-K points closest to a given lat/lon)
      searches. While the previous implementation appends all matching points to the
      geoPoint array and performs pruning after-the-fact via a full sort and [0, count)-based
      for-loop, this PR sorts only the required number of elements.
      
      This optimization provides a 5-20% improvement in runtime depending on the
      density of points of interest (POI) as well as the radius searched.
      No performance degradation has been observed.
      a3718cde
    • Huang Zw's avatar
      Cleanup: addReplyAggregateLen and addReplyBulkLen remove redundant check (#8431) · 9760475a
      Huang Zw authored
      addReplyLongLongWithPrefix, has a check against negative length, and the code
      flow removed in this commit bypasses the check.
      addReplyAggregateLen has an assertion for negative length, but addReplyBulkLen
      does not, so this commit fixes theoretical case of access violation (probably
      unreachable though)
      9760475a
    • zhaozhao.zz's avatar
      XINFO should use lookupKeyReadOrReply (#8436) · f2a5fe36
      zhaozhao.zz authored
      This bug would have let users observe logically expired keys on replicas and
      during CLIENT PAUSE WRITE.
      f2a5fe36
  7. 31 Jan, 2021 3 commits
  8. 30 Jan, 2021 1 commit
    • Wen Hui's avatar
      fix sentinel tests error (#8422) · eacccd2a
      Wen Hui authored
      This commit fixes sentinel announces hostnames test error in certain linux environment
      Before this commit, we only check localhost is resolved into 127.0.0.1, however in ubuntu
      or some other linux environments "localhost" will be resolved into ::1 ipv6 address first if
      the network stack is capable.
      eacccd2a
  9. 29 Jan, 2021 3 commits
    • filipe oliveira's avatar
      Enabled background and reply time tracking on blocked on keys/blocked on... · f0c5052a
      filipe oliveira authored
      Enabled background and reply time tracking on blocked on keys/blocked on background work clients (#7491)
      
      This commit enables tracking time of the background tasks and on replies,
      opening the door for properly tracking commands that rely on blocking / background
       work via the slowlog, latency history, and commandstats. 
      
      Some notes:
      - The time spent blocked waiting for key changes, or blocked on synchronous
        replication is not accounted for. 
      
      - **This commit does not affect latency tracking of commands that are non-blocking
        or do not have background work.** ( meaning that it all stays the same with exception to
        `BZPOPMIN`,`BZPOPMAX`,`BRPOP`,`BLPOP`, etc... and module's commands that rely
        on background threads ). 
      
      -  Specifically for latency history command we've added a new event class named
        `command-unblocking` that will enable latency monitoring on commands that spawn
        background threads to do the work.
      
      - For blocking commands we're now considering the total time of a command as the
        time spent on call() + the time spent on replying when unblocked.
      
      - For Modules commands that rely on background threads we're now considering the
        total time of a command as the time spent on call (main thread) + the time spent on
        the background thread ( if marked within `RedisModule_MeasureTimeStart()` and
        `RedisModule_MeasureTimeEnd()` ) + the time spent on replying (main thread)
      
      To test for this feature we've added a `unit/moduleapi/blockonbackground` test that relies on
      a module that blocks the client and sleeps on the background for a given time. 
      - check blocked command that uses RedisModule_MeasureTimeStart() is tracking background time
      - check blocked command that uses RedisModule_MeasureTimeStart() is tracking background time even in timeout
      - check blocked command with multiple calls RedisModule_MeasureTimeStart()  is tracking the total background time
      - check blocked command without calling RedisModule_MeasureTimeStart() is not reporting background time
      f0c5052a
    • Yang Bodong's avatar
      Add HRANDFIELD and ZRANDMEMBER. improvements to SRANDMEMBER (#8297) · b9a0500f
      Yang Bodong authored
      
      
      New commands:
      `HRANDFIELD [<count> [WITHVALUES]]`
      `ZRANDMEMBER [<count> [WITHSCORES]]`
      Algorithms are similar to the one in SRANDMEMBER.
      
      Both return a simple bulk response when no arguments are given, and an array otherwise.
      In case values/scores are requested, RESP2 returns a long array, and RESP3 a nested array.
      note: in all 3 commands, the only option that also provides random order is the one with negative count.
      
      Changes to SRANDMEMBER
      * Optimization when count is 1, we can use the more efficient algorithm of non-unique random
      * optimization: work with sds strings rather than robj
      
      Other changes:
      * zzlGetScore: when zset needs to convert string to double, we use safer memcpy (in
        case the buffer is too small)
      * Solve a "bug" in SRANDMEMBER test: it intended to test a positive count (case 3 or
        case 4) and by accident used a negative count
      Co-authored-by: default avatarxinluton <xinluton@qq.com>
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      b9a0500f
    • zhaozhao.zz's avatar
      AOF: recover from last write error after turn on appendonly again (#8030) · 49b36633
      zhaozhao.zz authored
      The key point is how to recover from last AOF write error, for example:
      
      1. start redis with appendonly yes, and append some write commands
      
      2. short write or something else error happen, `server.aof_last_write_status` changed to `C_ERR`, now redis doesn't accept write commands
      
      3. execute `CONFIG SET appendonly no` to avoid the above problem, now redis can accept write commands again
      
      4. disk error resolved, and execute `CONFIG SET appendonly yes` to reopen AOF, but `server.aof_last_write_status` cannot be changed to `C_OK` (if background aof rewrite run less then 1 second, it will free `server.aof_buf` and then serverCron cannot fix `aof_last_write_status`), then redis cannot accept write commands forever.
      
      This PR use a simple way to fix it:
      
      1. just free `server.aof_buf` when stop appendonly to save memory, if error happens in `flushAppendOnlyFile(1)`, the `server.aof_buf` may contains some data which has not be written to aof, I think we can ignore it because we turn off the appendonly.
      
      2. reset fsync status after stop appendonly and call `flushAppendOnlyFile` only when `aof_state` is ON
      
      3. reset `server.last_write_status` when reopen aof to accept write commands
      49b36633
  10. 28 Jan, 2021 9 commits
    • Allen Farris's avatar
      implement FAILOVER command (#8315) · 0d18a1e8
      Allen Farris authored
      Implement FAILOVER command, which coordinates failover
      between the server and one of its replicas.
      0d18a1e8
    • Yossi Gottlieb's avatar
      Update CLIENT HELP regarding KILL options. (#8417) · 26301897
      Yossi Gottlieb authored
      * Indicate address can also be a unix socket path name.
      * Document the LADDR option as well.
      26301897
    • Yossi Gottlieb's avatar
      Add proc-title-template option. (#8397) · 4bb5ccbe
      Yossi Gottlieb authored
      Make it possible to customize the process title, i.e. include custom
      strings, immutable configuration like port, tls-port, unix socket name,
      etc.
      4bb5ccbe
    • guybe7's avatar
      Modules: Add event for fork child birth and termination (#8289) · 01cbf17b
      guybe7 authored
      Useful to avoid doing background jobs that can cause excessive COW
      01cbf17b
    • Wang Yuan's avatar
      Redis exit for fsync error when the AOF fsync policy is 'always' (#8347) · a16739a3
      Wang Yuan authored
      With AOF policy of fsync "always", redis should respect the contract with the user
      that on acknowledged write data is already synced on disk.
      
      Redis was already exiting for AOF write error,  but don't care about fsync failure.
      So to guarantee data safe, redis should exit for fsync error too (when the AOF fsync
      policy is 'always').
      a16739a3
    • Viktor Söderqvist's avatar
      Add modules API for streams (#8288) · 4355145a
      Viktor Söderqvist authored
      APIs added for these stream operations: add, delete, iterate and
      trim (by ID or maxlength). The functions are prefixed by RM_Stream.
      
      * RM_StreamAdd
      * RM_StreamDelete
      * RM_StreamIteratorStart
      * RM_StreamIteratorStop
      * RM_StreamIteratorNextID
      * RM_StreamIteratorNextField
      * RM_StreamIteratorDelete
      * RM_StreamTrimByLength
      * RM_StreamTrimByID
      
      The type RedisModuleStreamID is added and functions for converting
      from and to RedisModuleString.
      
      * RM_CreateStringFromStreamID
      * RM_StringToStreamID
      
      Whenever the stream functions return REDISMODULE_ERR, errno is set to
      provide additional error information.
      
      Refactoring: The zset iterator fields in the RedisModuleKey struct
      are wrapped in a union, to allow the same space to be used for type-
      specific info for streams and allow future use for other key types.
      4355145a
    • Yossi Gottlieb's avatar
      Add hostname support in Sentinel. (#8282) · bb7cd974
      Yossi Gottlieb authored
      
      
      This is both a bugfix and an enhancement.
      
      Internally, Sentinel relies entirely on IP addresses to identify
      instances. When configured with a new master, it also requires users to
      specify and IP and not hostname.
      
      However, replicas may use the replica-announce-ip configuration to
      announce a hostname. When that happens, Sentinel fails to match the
      announced hostname with the expected IP and considers that a different
      instance, triggering reconfiguration, etc.
      
      Another use case is where TLS is used and clients are expected to match
      the hostname to connect to with the certificate's SAN attribute. To
      properly implement this configuration, it is necessary for Sentinel to
      redirect clients to a hostname rather than an IP address.
      
      The new 'resolve-hostnames' configuration parameter determines if
      Sentinel is willing to accept hostnames. It is set by default to no,
      which maintains backwards compatibility and avoids unexpected DNS
      resolution delays on systems with DNS configuration issues.
      
      Internally, Sentinel continues to identify instances by their resolved
      IP address and will also report the IP by default. The new
      'announce-hostnames' parameter determines if Sentinel should prefer to
      announce a hostname, when available, rather than an IP address. This
      applies to addresses returned to clients, as well as their
      representation in the configuration file, REPLICAOF configuration
      commands, etc.
      
      This commit also introduces SENTINEL CONFIG GET and SENTINEL CONFIG SET
      which can be used to introspect or configure global Sentinel
      configuration that was previously was only possible by directly
      accessing the configuration file and possibly restarting the instance.
      Co-authored-by: default avatarmyl1024 <myl92916@qq.com>
      Co-authored-by: default avatarsundb <sundbcn@gmail.com>
      bb7cd974
    • Z. Liu's avatar
      Add 'set-proc-title' config so that this mechanism can be disabled (#3623) · 17b34c73
      Z. Liu authored
      
      
      if option `set-proc-title' is no, then do nothing for proc title.
      
      The reason has been explained long ago, see following:
      
      We update redis to 2.8.8, then found there are some side effect when
      redis always change the process title.
      
      We run several slave instance on one computer, and all these salves
      listen on unix socket only, then ps will show:
      
        1 S redis 18036 1 0 80 0 - 56130 ep_pol 14:02 ? 00:00:31 /usr/sbin/redis-server *:0
        1 S redis 23949 1 0 80 0 - 11074 ep_pol 15:41 ? 00:00:00 /usr/sbin/redis-server *:0
      
      for redis 2.6 the output of ps is like following:
      
        1 S redis 18036 1 0 80 0 - 56130 ep_pol 14:02 ? 00:00:31 /usr/sbin/redis-server /etc/redis/a.conf
        1 S redis 23949 1 0 80 0 - 11074 ep_pol 15:41 ? 00:00:00 /usr/sbin/redis-server /etc/redis/b.conf
      
      Later is more informational in our case. The situation
      is worse when we manage the config and process running
      state by salt. Salt check the process by running "ps |
      grep SIG" (for Gentoo System) to check the running
      state, where SIG is the string to search for when
      looking for the service process with ps. Previously, we
      define sig as "/usr/sbin/redis-server
      /etc/redis/a.conf". Since the ps output is identical for
      our case, so we have no way to check the state of
      specified redis instance.
      
      So, for our case, we prefer the old behavior, i.e, do
      not change the process title for the main redis process.
      Or add an option such as "set-proc-title [yes|no]" to
      control this behavior.
      Co-authored-by: default avatarYossi Gottlieb <yossigo@gmail.com>
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      17b34c73
    • Yossi Gottlieb's avatar
      Avoid assertions when testing arm64 cow bug. (#8405) · 3a504904
      Yossi Gottlieb authored
      At least in one case the arm64 cow kernel bug test triggers an assert, which is a problem because it cannot be ignored like cases where the bug is found.
      
      On older systems (Linux <4.5) madvise fails because MADV_FREE is not supported. We treat these failures as an indication the system is not affected.
      
      Fixes #8351, #8406
      3a504904
  11. 27 Jan, 2021 3 commits
    • Raghav Muddur's avatar
      GETEX, GETDEL and SET PXAT/EXAT (#8327) · 0367a808
      Raghav Muddur authored
      This commit introduces two new command and two options for an existing command
      
      GETEX <key> [PERSIST][EX seconds][PX milliseconds] [EXAT seconds-timestamp]
      [PXAT milliseconds-timestamp]
      
      The getexCommand() function implements extended options and variants of the GET
      command. Unlike GET command this command is not read-only. Only one of the options
      can be used at a given time.
      
      1. PERSIST removes any TTL associated with the key.
      2. EX Set expiry TTL in seconds.
      3. PX Set expiry TTL in milliseconds.
      4. EXAT Same like EX instead of specifying the number of seconds representing the
          TTL (time to live), it takes an absolute Unix timestamp
      5. PXAT Same like PX instead of specifying the number of milliseconds representing the
          TTL (time to live), it takes an absolute Unix timestamp
      
      Command would return either the bulk string, error or nil.
      
      GETDEL <key>
      Would delete the key after getting.
      
      SET key value [NX] [XX] [KEEPTTL] [GET] [EX <seconds>] [PX <milliseconds>]
      [EXAT <seconds-timestamp>][PXAT <milliseconds-timestamp>]
      
      Two new options added here are EXAT and PXAT
      
      Key implementation notes
      - `SET` with `PX/EX/EXAT/PXAT` is always translated to `PXAT` in `AOF`. When relative time is
        specified (`PX/EX`), replication will always use `PX`.
      - `setexCommand` and `psetexCommand` would no longer need translation in `feedAppendOnlyFile`
        as they are modified to invoke `setGenericCommand ` with appropriate flags which will take care of
        correct AOF translation.
      - `GETEX` without any optional argument behaves like `GET`.
      - `GETEX` command is never propagated, It is either propagated as `PEXPIRE[AT], or PERSIST`.
      - `GETDEL` command is propagated as `DEL`
      - Combined the validation for `SET` and `GETEX` arguments. 
      - Test cases to validate AOF/Replication propagation
      0367a808
    • Huang Zw's avatar
      Fix bug in activeDefragSdsListAndDict dead code (#8403) · f395119e
      Huang Zw authored
      In activeDefragSdsListAndDict when dict_val_type is DEFRAG_SDS_DICT_VAL_VOID_PTR, it should update de->v.val not ln->value.
      Because this code path will never be executed, so this bug never happened.
      f395119e
    • Wang Yuan's avatar
      Implement rdb-only replication (#8303) · ad7d4c6b
      Wang Yuan authored
      In some scenarios, such as remote backup, we only want to get remote
      redis server db snapshot. Currently, redis-cli acts as a replica and
      sends SYNC to redis, but redis still accumulates replication buffer
      in the replica client output buffer, that may result in using vast
      memory, or failing to transfer RDB because of client-output-buffer-limit.
      In this commit, we add 'replconf rdb-only 0|1', redis doesn't send
      incremental replication buffer to them if they send 'replconf rdb-only 1',
      so we can reduce used memory and improve success of getting RDB.
      ad7d4c6b
  12. 26 Jan, 2021 1 commit