1. 28 Jan, 2021 1 commit
    • Yossi Gottlieb's avatar
      Avoid assertions when testing arm64 cow bug. (#8405) · 3a504904
      Yossi Gottlieb authored
      At least in one case the arm64 cow kernel bug test triggers an assert, which is a problem because it cannot be ignored like cases where the bug is found.
      
      On older systems (Linux <4.5) madvise fails because MADV_FREE is not supported. We treat these failures as an indication the system is not affected.
      
      Fixes #8351, #8406
      3a504904
  2. 27 Jan, 2021 3 commits
    • Raghav Muddur's avatar
      GETEX, GETDEL and SET PXAT/EXAT (#8327) · 0367a808
      Raghav Muddur authored
      This commit introduces two new command and two options for an existing command
      
      GETEX <key> [PERSIST][EX seconds][PX milliseconds] [EXAT seconds-timestamp]
      [PXAT milliseconds-timestamp]
      
      The getexCommand() function implements extended options and variants of the GET
      command. Unlike GET command this command is not read-only. Only one of the options
      can be used at a given time.
      
      1. PERSIST removes any TTL associated with the key.
      2. EX Set expiry TTL in seconds.
      3. PX Set expiry TTL in milliseconds.
      4. EXAT Same like EX instead of specifying the number of seconds representing the
          TTL (time to live), it takes an absolute Unix timestamp
      5. PXAT Same like PX instead of specifying the number of milliseconds representing the
          TTL (time to live), it takes an absolute Unix timestamp
      
      Command would return either the bulk string, error or nil.
      
      GETDEL <key>
      Would delete the key after getting.
      
      SET key value [NX] [XX] [KEEPTTL] [GET] [EX <seconds>] [PX <milliseconds>]
      [EXAT <seconds-timestamp>][PXAT <milliseconds-timestamp>]
      
      Two new options added here are EXAT and PXAT
      
      Key implementation notes
      - `SET` with `PX/EX/EXAT/PXAT` is always translated to `PXAT` in `AOF`. When relative time is
        specified (`PX/EX`), replication will always use `PX`.
      - `setexCommand` and `psetexCommand` would no longer need translation in `feedAppendOnlyFile`
        as they are modified to invoke `setGenericCommand ` with appropriate flags which will take care of
        correct AOF translation.
      - `GETEX` without any optional argument behaves like `GET`.
      - `GETEX` command is never propagated, It is either propagated as `PEXPIRE[AT], or PERSIST`.
      - `GETDEL` command is propagated as `DEL`
      - Combined the validation for `SET` and `GETEX` arguments. 
      - Test cases to validate AOF/Replication propagation
      0367a808
    • Huang Zw's avatar
      Fix bug in activeDefragSdsListAndDict dead code (#8403) · f395119e
      Huang Zw authored
      In activeDefragSdsListAndDict when dict_val_type is DEFRAG_SDS_DICT_VAL_VOID_PTR, it should update de->v.val not ln->value.
      Because this code path will never be executed, so this bug never happened.
      f395119e
    • Wang Yuan's avatar
      Implement rdb-only replication (#8303) · ad7d4c6b
      Wang Yuan authored
      In some scenarios, such as remote backup, we only want to get remote
      redis server db snapshot. Currently, redis-cli acts as a replica and
      sends SYNC to redis, but redis still accumulates replication buffer
      in the replica client output buffer, that may result in using vast
      memory, or failing to transfer RDB because of client-output-buffer-limit.
      In this commit, we add 'replconf rdb-only 0|1', redis doesn't send
      incremental replication buffer to them if they send 'replconf rdb-only 1',
      so we can reduce used memory and improve success of getting RDB.
      ad7d4c6b
  3. 26 Jan, 2021 4 commits
    • Oran Agra's avatar
      Add tests for RESP3 responce of ZINTER and ZRANGE (#8391) · 9e56d396
      Oran Agra authored
      It was confusing as to why these don't return a map type.
      the reason is that order matters, so we need to make sure the client
      library knows to respect it.
      Added comments in the implementation and tests to cover it.
      9e56d396
    • Wen Hui's avatar
      Sentinel: Fix Config Dependency and Rewrite Sequence (#8271) · 1aad55b6
      Wen Hui authored
      This commit fixes a well known and an annoying issue in Sentinel mode.
      
      Cause of this issue:
      Currently, Redis rewrite process works well in server mode, however in sentinel mode,
      the sentinel config has variant semantics for different configurations, in example configuration
      https://github.com/redis/redis/blob/unstable/sentinel.conf, we put comments on these.
      However the rewrite process only treat the sentinel config as a single option. During rewrite
      process, it will mess up with the lines and comments.
      
      Approaches:
      In order to solve this issue, we need to differentiate different subconfig options in sentinel separately,
      for example, sentinel monitor <master-name> <ip> <redis-port> <quorum>
      we can treat it as sentinel monitor option, instead of the sentinel option.
      
      This commit also fixes the dependency issue when putting configurations in sentinel.conf.
      For example before this commit,we must put
      `sentinel monitor <master-name> <ip> <redis-port> <quorum>` before
      `sentinel auth-pass <master-name> <password>` for a single master,
      otherwise the server cannot start and will return error. This commit fixes this issue, as long as
      the monitoring master was configured, no matter the sequence is, the sentinel can start and run properly.
      1aad55b6
    • Vladimir Maksimovski's avatar
      Remove check leading to duplicate branches (#8398) · 70789cf4
      Vladimir Maksimovski authored
      Remove check leading to duplicate branches and unused withscores parameter
      70789cf4
    • zhaozhao.zz's avatar
      latency: add expire-del event (#8394) · 951d25ff
      zhaozhao.zz authored
      951d25ff
  4. 22 Jan, 2021 1 commit
  5. 21 Jan, 2021 3 commits
  6. 20 Jan, 2021 4 commits
    • Yossi Gottlieb's avatar
      CONFIG REWRITE should honor umask settings. (#8371) · b548ffab
      Yossi Gottlieb authored
      Fixes a regression introduced due to a new (safer) way of rewriting configuration files. In the past the file was simply overwritten (same inode), but now Redis creates a new temporary file and later renames it over the old one.
      
      The temp file typically gets created with 0600 permissions so we later fchmod it to fix that. Unlike open with O_CREAT, fchmod doesn't consider umask so we have to do that explicitly.
      
      Fixes #8369
      b548ffab
    • Guy Korland's avatar
      Add CI for FreeBSD (#8292) · ac5f21d6
      Guy Korland authored
      
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      ac5f21d6
    • guybe7's avatar
      Fix firstkey,lastkey,step in COMMAND command for some commands (#8367) · baf92f3f
      guybe7 authored
      The output for COMMAND command was wrong for some commands.
      clients can use firstkey,lastkey,step to find (some) key name arguments, and the
      "movablekeys" flag to know that they can't know all (or any) of the key name arguments.
      
      These commands had the wrong output:
      1. GEORADIUS*_RO used to have "movablekeys" (which it doesn't really need)
      2. XREAD and XREADGROUP used to have (1,1,1). but that's completely wrong.
      3. Z*STORE used to have (0,0,0) but it can at lest give the index of the dstkey (1,1,1)
      baf92f3f
    • Viktor Söderqvist's avatar
      More modules API ref formatting fixes (#8344) · 16258f21
      Viktor Söderqvist authored
      Fix broken formatting in `RM_Call` and `RM_CreateDataType`,
      `RM_SubscribeToServerEvent` (nested lists, etc. in list items).
      
      Unhide docs of `RM_LoadDataTypeFromString` and
      `RM_SaveDataTypeToString` by removing blank line between docs and
      function.
      
      Clarification added to `RM__Assert`: Recommentation to use the
      `RedisModule_Assert` macro instead.
      
      All names containing underscores (variable and macro names) are
      wrapped in backticks (if not already wrapped in backticks). This
      prevents underscore from being interpreted as italics in some
      cases.
      
      Names including a wildcard star, e.g. RM_Defrag*(), is wrapped in
      backticks (and RM replaced by RedisModule in this case). This
      prevents the * from being interpreted as an italics marker.
      
      A list item with a sublist, a paragraph and another sublist is a
      combination which seems impossible to achieve with RedCarped
      markdown, so the one occurrence of this is rewritten.
      
      Various trivial changes (typos, backticks, etc.).
      
      Ruby script:
      
      * Replace `RM_Xyz` with `RedisModule_Xyz` in docs. (RM is correct
        when refering to the C code but RedisModule is correct in the
        API docs.)
      * Automatic backquotes around C functions like `malloc()`.
      * Turn URLs into links. The link text is the URL itself.
      * Don't add backticks inside bold (**...**)
      16258f21
  7. 19 Jan, 2021 4 commits
  8. 18 Jan, 2021 1 commit
  9. 17 Jan, 2021 1 commit
  10. 15 Jan, 2021 3 commits
    • Yang Bodong's avatar
      Add lazyfree-lazy-user-flush config to control default behavior of... · 294f93af
      Yang Bodong authored
      Add lazyfree-lazy-user-flush config to control default behavior of FLUSH[ALL|DB], SCRIPT FLUSH (#8258)
      
      * Adds ASYNC and SYNC arguments to SCRIPT FLUSH
      * Adds SYNC argument to FLUSHDB and FLUSHALL
      * Adds new config to control the default behavior of FLUSHDB, FLUSHALL and SCRIPT FLUASH.
      
      the new behavior is as follows:
      * FLUSH[ALL|DB],SCRIPT FLUSH: Determine sync or async according to the
        value of lazyfree-lazy-user-flush.
      * FLUSH[ALL|DB],SCRIPT FLUSH ASYNC: Always flushes the database in an async manner.
      * FLUSH[ALL|DB],SCRIPT FLUSH SYNC: Always flushes the database in a sync manner.
      294f93af
    • Viktor Söderqvist's avatar
      Rename non-API RM-prefixed functions to hide them from API docs · fcb3dfe5
      Viktor Söderqvist authored
      The prefix is changed from `RM_` to `module` on the following
      internal functions, to prevent them from appearing in the API docs:
      
          RM_LogRaw -> moduleLogRaw
          RM_FreeCallReplyRec -> moduleFreeCallReplyRec
          RM_ZsetAddFlagsToCoreFlags -> moduleZsetAddFlagsToCoreFlags
          RM_ZsetAddFlagsFromCoreFlags -> moduleZsetAddFlagsFromCoreFlags
      fcb3dfe5
    • Viktor Söderqvist's avatar
      Modules API reference formatting fixes · ebf20b83
      Viktor Söderqvist authored
      Fixes markdown formatting errors and some functions not showing
      up in the generated documentation at all.
      
      Ruby script (gendoc.rb) fixes:
      
      * Modified automatic instertion of backquotes:
        * Don't add backquotes around names which are already preceded by a
          backquote. Fixes for example \`RedisModule_Reply\*\` which turning
          into \`\`RedisModule_Reply\`\*\` messes up the formatting.
        * Add backquotes around types such as RedisModuleString (in addition
          to function names `RedisModule_[A-z()]*` and macro names
          `REDISMODULE_[A-z]*`).
        * Require 4 spaces indentation for disabling automatic backquotes, i.e.
          code blocks. Fixes continuations of list items (indented 2 spaces).
      * More permissive extraction of doc comments:
        * Allow doc comments starting with `/**`.
        * Make space before `*` on each line optional.
        * Make space after `/*` and `/**` optional (needed when appearing on
          its own line).
      
      Markdown fixes in module.c:
      
      * Fix code blocks not indented enough (4 spaces needed).
      * Add black line before code blocks and lists where missing (needed).
      * Enclose special markdown characters `_*^<>` in backticks to prevent them
        from messing up formatting.
      * Lists with `1)` changed to `1.` for proper markdown lists.
      * Remove excessive indentation which causes text to be unintentionally
        rendered as code blocks.
      * Other minor formatting fixes.
      
      Other fixes in module.c:
      
      * Remove blank lines between doc comment and function definition. A blank
        line here makes the Ruby script exclude the function in docs.
      ebf20b83
  11. 14 Jan, 2021 1 commit
  12. 13 Jan, 2021 4 commits
    • Wang Yuan's avatar
      Optimize performance of clusterGenNodesDescription for large clusters (#8182) · 9cb9f98d
      Wang Yuan authored
      Optimize the performance of clusterGenNodesDescription by only checking slot ownership of each slot once, instead of checking each slot for each node. 
      9cb9f98d
    • houzj.fnst's avatar
      remove some unnecessary checks (#7431) · f5577fdb
      houzj.fnst authored
      Remove several checks that always evaluate to true.
      f5577fdb
    • sundb's avatar
      Fix use of lookupKeyRead and lookupKeyWrite in zrangeGenericCommand,... · 8a81ed1b
      sundb authored
      Fix use of  lookupKeyRead and lookupKeyWrite in zrangeGenericCommand, zunionInterDiffGenericCommand (#8316)
      
      * Change zunionInterDiffGenericCommand to use lookupKeyRead if dstkey is null
      * Change zrangeGenericCommand to use lookupKey Write if dstkey isn't null
      
      ZRANGESTORE and UNION, ZINTER, ZDIFF are all new commands (6.2 RC1 and RC2).
      In redis 6.0 the ZRANGE was using lookupKeyRead, and ZUNIONSTORE / ZINTERSTORE were using lookupKeyWrite.
      So there bugs are introduced in 6.2 and will be resolved before it is released.
      
      the implications of this bug are also not big:
      The sole difference between LookupKeyRead and LookupKeyWrite is for command executed on a replica, which are not received from its master client. (for the master, and for the master client on the replica, these two functions behave the same)!
      8a81ed1b
    • sundb's avatar
      Fix units in log message of copy-on-write (#8320) · 593cde16
      sundb authored
      593cde16
  13. 12 Jan, 2021 2 commits
    • Madelyn Olson's avatar
      Fix issues in wait test (#8310) · b24b4903
      Madelyn Olson authored
      This fixes three issues:
      1.  Using debug SLEEP was impacting the subsequent test, and causing it to pass reliably even though it should have failed. There was exactly 5 seconds of artificial pause (after 1000, wait 3000, wait 1000) between the debug sleep 5 and when we needed to unblock the client in the subsequent test. Now the test properly makes sure the client is unblocked, and the subsequent test is fixed.
      2. Minor, the client pause types were using & comparisons instead of ==, since it was previously a flag.
      3. Test is faster now that some of the hand wavy time is removed. 
      b24b4903
    • Bob Li's avatar
      Fix overflow of rdbWriteRaw return value (#8306) · 542455ce
      Bob Li authored
      Saving string of more than 2GB to the RDB file, can result in corrupt RDB, or failure in rdbSave.
      S
      542455ce
  14. 09 Jan, 2021 3 commits
  15. 08 Jan, 2021 5 commits
    • Oran Agra's avatar
      Fix last COW INFO report, Skip test on non-linux platforms (#8301) · 8dd16cae
      Oran Agra authored
      - the last COW report wasn't always read from the pipe
        (receiveLastChildInfo wasn't used)
      - but in fact, there's no reason we won't always try to drain that pipe
        so i'm unifying receiveLastChildInfo with receiveChildInfo
      - adjust threshold of the COW test when run in accurate mode
      - add some prints in case this test fails again
      - fix indentation, page size, and PID! in MacOS proc info
      
      p.s. it seems that pri_pages_dirtied is always 0
      8dd16cae
    • Yang Bodong's avatar
      GEOSEARCH - ANY option, for limited search that returns ASAP (#8259) · ea5350c5
      Yang Bodong authored
      Support ANY option to return some results that match the criteria ASAP,
      without a complete search and implicit sorting.
      ea5350c5
    • guybe7's avatar
      XADD and XTRIM, Trim by MINID, and new LIMIT argument (#8169) · 814aad65
      guybe7 authored
      This PR adds another trimming strategy to XADD and XTRIM named MINID
      (complements the existing MAXLEN).
      It also adds a new LIMIT argument that allows incremental trimming by repeated
      calls (rather than all at once).
      
      This provides the ability to trim all records older than a certain ID (which makes it
      possible for the user to trim by age too).
      Example:
      XTRIM mystream MINID ~ 1608540753 will trim entries with id < 1608540753,
      but might not trim all (because of the ~ modifier)
      
      The purpose is to ease the use of streams. many users use streams as logs and
      the common case is wanting a log
      of the last X seconds rather than a log that contains maximum X entries (new
      MINID vs existing MAXLEN)
      
      The new LIMIT modifier is only supported when the trim strategy uses ~.
      i.e. when the user asked for exact trimming, it all happens in one go (no
      possibility for incremental trimming).
      However, when ~ is provided, we trim full rax nodes, up to the limit number
      of records.
      The default limit is 100*stream_node_max_entries (used when LIMIT is not
      provided).
      I.e. this is a behavior change (even if the existing MAXLEN strategy is used).
      An explicit limit of 0 means unlimited (but note that it's not the default).
      
      Other changes:
      
      Refactor arg parsing code for XADD and XTRIM to use common code.
      814aad65
    • Madelyn Olson's avatar
      Throw error for conflicting bcast tracking prefixes (#8176) · 999494ce
      Madelyn Olson authored
      Throw an error if there are conflicting bcast tracking prefixes. 
      999494ce
    • Madelyn Olson's avatar
      Add support for client pause WRITE (#8170) · 47579bdf
      Madelyn Olson authored
      Implementation of client pause WRITE and client unpause
      47579bdf