1. 04 Jan, 2022 5 commits
    • Matthieu MOREL's avatar
      Setup dependabot for github-actions and codespell (#9857) · d5a3b3f5
      Matthieu MOREL authored
      
      
      This sets up  dependabot to check weekly updates for pip and github-actions dependencies.
      If it finds an update it will create a PR to update the dependency. More information can be found here
      
      It includes the update of:
      
      * vmactions/freebsd-vm from 0.1.4 to 0.1.5
      * codespell from 2.0.0 to 2.1.0
      
      Also includes spelling fixes found by the latest version of codespell.
      Includes a dedicated .codespell folder so dependabot can read a requirements.txt file and every files dedicated to codespell can be grouped in the same place
      Co-Authored-By: default avatarMatthieu MOREL <mmorel-35@users.noreply.github.com>
      Co-Authored-By: default avatarMOREL Matthieu <matthieu.morel@cnp.fr>
      d5a3b3f5
    • Binbin's avatar
      Print error messages in monitor/pubsub when errors occurs (#10050) · c57e41c0
      Binbin authored
      In monitor/pubsub mode, if the server closes the connection,
      for example, use `CLIENT KILL`, redis-cli will exit directly
      without printing any error messages.
      
      This commit ensures that redis-cli will try to print the
      error messages before exiting. Also there is a minor cleanup
      for restart, see the example below.
      
      before:
      ```
      127.0.0.1:6379> monitor
      OK
      [root@ redis]#
      
      127.0.0.1:6379> subscribe channel
      Reading messages... (press Ctrl-C to quit)
      1) "subscribe"
      2) "channel"
      3) (integer) 1
      [root@ redis]#
      
      127.0.0.1:6379> restart
      127.0.0.1:6379> get keyUse 'restart' only in Lua debugging mode.
      (nil)
      ```
      
      after:
      ```
      127.0.0.1:6379> monitor
      OK
      Error: Server closed the connection
      [root@ redis]#
      
      127.0.0.1:6379> subscribe channel
      Reading messages... (press Ctrl-C to quit)
      1) "subscribe"
      2) "channel"
      3) (integer) 1
      Error: Server closed the connection
      [root@ redis]#
      
      127.0.0.1:6379> restart
      Use 'restart' only in Lua debugging mode.
      ```
      c57e41c0
    • 王辉's avatar
      Fix C11_ATOMIC detection on GNU Make 4.3 (#10033) · 747b08be
      王辉 authored
      Older version of GNU Make (<4.3) required quoting of number signs (#) to
      avoid them being treated as a comment. Newer versions will treat this
      quote as a literal.
      
      This issue and a proposed solution is discussed here:
      https://lists.gnu.org/archive/html/info-gnu/2020-01/msg00004.html
      
      Co-authored-by: default avatarYossi Gottlieb <yossigo@gmail.com>
      747b08be
    • guybe7's avatar
      Ban snapshot-creating commands and other admin commands from transactions (#10015) · ac84b1cd
      guybe7 authored
      
      
      Creating fork (or even a foreground SAVE) during a transaction breaks the atomicity of the transaction.
      In addition to that, it could mess up the propagated transaction to the AOF file.
      
      This change blocks SAVE, PSYNC, SYNC and SHUTDOWN from being executed inside MULTI-EXEC.
      It does that by adding a command flag, so that modules can flag their commands with that flag too.
      
      Besides it changes BGSAVE, BGREWRITEAOF, and CONFIG SET appendonly, to turn the
      scheduled flag instead of forking righ taway.
      
      Other changes:
      * expose `protected`, `no-async-loading`, and `no_multi` flags in COMMAND command
      * add a test to validate propagation of FLUSHALL inside a transaction.
      * add a test to validate how CONFIG SET that errors reacts in a transaction
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      ac84b1cd
    • zhaozhao.zz's avatar
      use startEvictionTimeProc() in config set maxmemory (#10019) · 2e1979a2
      zhaozhao.zz authored
      This would mean that the effects of `CONFIG SET maxmemory` may not be visible once the command returns.
      That could anyway happen since incremental eviction was added in redis 6.2 (see #7653)
      
      We do this to fix one of the propagation bugs about eviction see #9890 and #10014.
      2e1979a2
  2. 03 Jan, 2022 4 commits
    • chenyang8094's avatar
      Implement Multi Part AOF mechanism to avoid AOFRW overheads. (#9788) · 87789fae
      chenyang8094 authored
      
      
      Implement Multi-Part AOF mechanism to avoid overheads during AOFRW.
      Introducing a folder with multiple AOF files tracked by a manifest file.
      
      The main issues with the the original AOFRW mechanism are:
      * buffering of commands that are processed during rewrite (consuming a lot of RAM)
      * freezes of the main process when the AOFRW completes to drain the remaining part of the buffer and fsync it.
      * double disk IO for the data that arrives during AOFRW (had to be written to both the old and new AOF files)
      
      The main modifications of this PR:
      1. Remove the AOF rewrite buffer and related code.
      2. Divide the AOF into multiple files, they are classified as two types, one is the the `BASE` type,
        it represents the full amount of data (Maybe AOF or RDB format) after each AOFRW, there is only
        one `BASE` file at most. The second is `INCR` type, may have more than one. They represent the
        incremental commands since the last AOFRW.
      3. Use a AOF manifest file to record and manage these AOF files mentioned above.
      4. The original configuration of `appendfilename` will be the base part of the new file name, for example:
        `appendonly.aof.1.base.rdb` and `appendonly.aof.2.incr.aof`
      5. Add manifest-related TCL tests, and modified some existing tests that depend on the `appendfilename`
      6. Remove the `aof_rewrite_buffer_length` field in info.
      7. Add `aof-disable-auto-gc` configuration. By default we're automatically deleting HISTORY type AOFs.
        It also gives users the opportunity to preserve the history AOFs. just for testing use now.
      8. Add AOFRW limiting measure. When the AOFRW failures reaches the threshold (3 times now),
        we will delay the execution of the next AOFRW by 1 minute. If the next AOFRW also fails, it will be
        delayed by 2 minutes. The next is 4, 8, 16, the maximum delay is 60 minutes (1 hour). During the limit
        period, we can still use the 'bgrewriteaof' command to execute AOFRW immediately.
      9. Support upgrade (load) data from old version redis.
      10. Add `appenddirname` configuration, as the directory name of the append only files. All AOF files and
        manifest file will be placed in this directory.
      11. Only the last AOF file (BASE or INCR) can be truncated. Otherwise redis will exit even if
        `aof-load-truncated` is enabled.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      87789fae
    • Meir Shpilraien (Spielrein)'s avatar
      Fix OOM error not raised of functions (#10048) · 78a62c01
      Meir Shpilraien (Spielrein) authored
      OOM Error did not raise on functions due to a bug.
      Added test to verify the fix.
      78a62c01
    • Madelyn Olson's avatar
      Implement clusterbus message extensions and cluster hostname support (#9530) · 5460c100
      Madelyn Olson authored
      Implement the ability for cluster nodes to advertise their location with extension messages.
      5460c100
    • Harkrishn Patro's avatar
      Sharded pubsub implementation (#8621) · 9f888576
      Harkrishn Patro authored
      
      
      This commit implements a sharded pubsub implementation based off of shard channels.
      Co-authored-by: default avatarHarkrishn Patro <harkrisp@amazon.com>
      Co-authored-by: default avatarMadelyn Olson <madelyneolson@gmail.com>
      9f888576
  3. 02 Jan, 2022 5 commits
    • Binbin's avatar
      Add DUMP RESTORE tests for redis-cli -x and -X options (#10041) · b8ba942a
      Binbin authored
      This commit adds DUMP RESTORES tests for the -x and -X options.
      I wanted to add it in #9980 which introduce the -X option, but
      back then i failed due to some errors (related to redis-cli call).
      b8ba942a
    • yoav-steinberg's avatar
      Make sure replicas don't write their own replies to the replication link (#10020) · 2ff3fc17
      yoav-steinberg authored
      Since #9166 we have an assertion here to make sure replica clients don't write anything to their buffer.
      But in reality a replica may attempt write data to it's buffer simply by sending a command on the replication link.
      This command in most cases will be rejected since #8868 but it'll still generate an error.
      Actually the only valid command to send on a replication link is 'REPCONF ACK` which generates no response.
      
      We want to keep the design so that replicas can send commands but we need to avoid any situation where we start
      putting data in their response buffers, especially since they aren't used anymore. This PR makes sure to disconnect
      a rogue client which generated a write on the replication link that cause something to be written to the response buffer.
      
      To recreate the bug this fixes simply connect via telnet to a redis server and write sync\r\n wait for the the payload to
      be written and then write any command (valid or invalid), such as ping\r\n on the telnet connection. It'll crash the server.
      2ff3fc17
    • Joey from AWS's avatar
      Report slot to keys map size in MEMORY STATS in cluster mode (#10017) · 09c668f2
      Joey from AWS authored
      Report slot to keys map size in MEMORY STATS in cluster mode
      Report dictMetadataSize in MEMORY USAGE command as well
      09c668f2
    • Viktor Söderqvist's avatar
      Wait for replicas when shutting down (#9872) · 45a155bd
      Viktor Söderqvist authored
      
      
      To avoid data loss, this commit adds a grace period for lagging replicas to
      catch up the replication offset.
      
      Done:
      
      * Wait for replicas when shutdown is triggered by SIGTERM and SIGINT.
      
      * Wait for replicas when shutdown is triggered by the SHUTDOWN command. A new
        blocked client type BLOCKED_SHUTDOWN is introduced, allowing multiple clients
        to call SHUTDOWN in parallel.
        Note that they don't expect a response unless an error happens and shutdown is aborted.
      
      * Log warning for each replica lagging behind when finishing shutdown.
      
      * CLIENT_PAUSE_WRITE while waiting for replicas.
      
      * Configurable grace period 'shutdown-timeout' in seconds (default 10).
      
      * New flags for the SHUTDOWN command:
      
          - NOW disables the grace period for lagging replicas.
      
          - FORCE ignores errors writing the RDB or AOF files which would normally
            prevent a shutdown.
      
          - ABORT cancels ongoing shutdown. Can't be combined with other flags.
      
      * New field in the output of the INFO command: 'shutdown_in_milliseconds'. The
        value is the remaining maximum time to wait for lagging replicas before
        finishing the shutdown. This field is present in the Server section **only**
        during shutdown.
      
      Not directly related:
      
      * When shutting down, if there is an AOF saving child, it is killed **even** if AOF
        is disabled. This can happen if BGREWRITEAOF is used when AOF is off.
      
      * Client pause now has end time and type (WRITE or ALL) per purpose. The
        different pause purposes are *CLIENT PAUSE command*, *failover* and
        *shutdown*. If clients are unpaused for one purpose, it doesn't affect client
        pause for other purposes. For example, the CLIENT UNPAUSE command doesn't
        affect client pause initiated by the failover or shutdown procedures. A completed
        failover or a failed shutdown doesn't unpause clients paused by the CLIENT
        PAUSE command.
      
      Notes:
      
      * DEBUG RESTART doesn't wait for replicas.
      
      * We already have a warning logged when a replica disconnects. This means that
        if any replica connection is lost during the shutdown, it is either logged as
        disconnected or as lagging at the time of exit.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      45a155bd
    • yoav-steinberg's avatar
      Generate RDB with Functions only via redis-cli --functions-rdb (#9968) · 1bf6d6f1
      yoav-steinberg authored
      
      
      This is needed in order to ease the deployment of functions for ephemeral cases, where user
      needs to spin up a server with functions pre-loaded.
      
      #### Details:
      
      * Added `--functions-rdb` option to _redis-cli_.
      * Functions only rdb via `REPLCONF rdb-filter-only functions`. This is a placeholder for a space
        separated inclusion filter for the RDB. In the future can be `REPLCONF rdb-filter-only
        "functions db:3 key-patten:user*"` and a complementing `rdb-filter-exclude` `REPLCONF`
        can also be added.
      * Handle "slave requirements" specification to RDB saving code so we can use the same RDB
        when different slaves express the same requirements (like functions-only) and not share the
        RDB when their requirements differ. This is currently just a flags `int`, but can be extended to
        a more complex structure with various filter fields.
      * make sure to support filters only in diskless replication mode (not to override the persistence file),
        we do that by forcing diskless (even if disabled by config)
      
      other changes:
      * some refactoring in rdb.c (extract portion of a big function to a sub-function)
      * rdb_key_save_delay used in AOFRW too
      * sendChildInfo takes the number of updated keys (incremental, rather than absolute)
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      1bf6d6f1
  4. 01 Jan, 2022 1 commit
    • sundb's avatar
      Fix a valgrind test failure due to slowly shutdown (#10038) · 888e92eb
      sundb authored
      This pr is mainly to solve the problem that redis process cannot be exited normally, due to changes in #10003.
      When a test uses the `key-load-delay` config to delay loading, but does not reset it at the end of the test, will lead to server wait for the loading to reach the event
      loop (once in 2mb) before actually shutting down.
      888e92eb
  5. 31 Dec, 2021 2 commits
  6. 30 Dec, 2021 3 commits
    • Viktor Söderqvist's avatar
      Modules: Mark all APIs non-experimental (#9983) · e4b3a257
      Viktor Söderqvist authored
      These exist for quite some time, and are no longer experimental
      e4b3a257
    • Binbin's avatar
      redis-cli: Add -X option and extend --cluster call take arg from stdin (#9980) · 4836ae32
      Binbin authored
      There are two changes in this commit:
      
      1. Add -X option to redis-cli.
      Currently `-x` can only be used to provide the last argument,
      so you can do `redis-cli dump keyname > key.dump`,
      and then do `redis-cli -x restore keyname 0 < key.dump`.
      
      But what if you want to add the replace argument (which comes last?).
      oran suggested adding such usage:
      `redis-cli -X <tag> restore keyname <tag> replace < key.dump`
      
      i.e. you're able to provide a string in the arguments that's gonna be
      substituted with the content from stdin.
      
      Note that the tag name should not conflict with others non-replaced args.
      And the -x and -X options are conflicting.
      
      Some usages:
      ```
      [root]# echo mypasswd | src/redis-cli -X passwd_tag mset username myname password passwd_tag                                                   OK
      [root]# echo username > username.txt
      [root]# head -c -1 username.txt | src/redis-cli -X name_tag mget name_tag password
      1) "myname"
      2) "mypasswd\n"
      ```
      
      2. Handle the combination of both `-x` and `--cluster` or `-X` and `--cluster`
      Extend the broadcast option to receive the last arg or <tag> arg from the stdin.
      
      Now we can use `redis-cli -x --cluster call <host>:<port> cmd`,
      or `redis-cli -X <tag> --cluster call <host>:<port> cmd <tag>`.
      (support part of #9899)
      4836ae32
    • Viktor Söderqvist's avatar
      Module API doc formatting from #9656 (#10026) · 5006eab5
      Viktor Söderqvist authored
      Add blank before lists, so that they will be rendered as lists.
      
      Commands RM_GetCommand and RM_CreateSubcommand were added in #9656.
      5006eab5
  7. 29 Dec, 2021 1 commit
    • Itamar Haber's avatar
      Add missing metadata to the commands SSOT files. (#10016) · aec8c577
      Itamar Haber authored
      Add missing information about commands, mainly from reviewing redis-doc and removing
      the metadata from it (https://github.com/redis/redis-doc/pull/1722)
      
      * Reintroduces CLUSTER S****S (supported by Redis) but missing from the JSON / docs (related? #9675).
        Note that without that json file, the command won't work (breaking change)
      * Adds the `replicas` argument (exists in Redis) to `CLIENT KILL`.
      * Adds `history` entries to several commands based on redis-doc's man pages.
      * Adds `since` to applicable command arguments based on `history` (this basically makes
        some of `history` redundant - perhaps at a later stage).
      * Uses proper semantic versioning in all version references.
      * Also removes `geoencodeCommand` and `geodecodeCommand` header
        declarations per b96af595.
      aec8c577
  8. 28 Dec, 2021 4 commits
  9. 27 Dec, 2021 4 commits
    • chenyang8094's avatar
      Tests: don't rely on the response of MEMORY USAGE when mem_allocator is not jemalloc (#10010) · af0b50f8
      chenyang8094 authored
      
      
      It turns out that libc malloc can return an allocation of a different size on requests of the same size.
      this means that matching MEMORY USAGE of one key to another copy of the same data can fail.
      
      Solution:
      Keep running the test that calls MEMORY USAGE, but ignore the response.
      We do that by introducing a new utility function to get the memory usage, which always returns 1
      when the allocator is not jemalloc.
      
      Other changes:
      Some formatting for datatype2.tcl
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      af0b50f8
    • Itamar Haber's avatar
      Adds utils/gen-commands-json.py (#9958) · f810510b
      Itamar Haber authored
      Following #9656, this script generates a "commands.json" file from the output
      of the new COMMAND. The output of this script is used in redis/redis-doc#1714
      and by redis/redis-io#259. This also converts a couple of rogue dashes (in 
      'key-specs' and 'multiple-token' flags) to underscores (continues #9959).
      f810510b
    • chenyang8094's avatar
      Fix failing test due to recent change in transaction propagation (#10006) · 317464a3
      chenyang8094 authored
      
      
      PR #9890 may have introduced a problem.
      There are tests that use MULTI-EXEC to make sure two BGSAVE / BGREWRITEAOF are executed together.
      But now it's not valid to run run commands that create a snapshot inside a transaction (gonna be blocked soon)
      This PR modifies the test not to rely on MULTI-EXEC.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      317464a3
    • guybe7's avatar
      Fix race in propagation test (#10012) · 0f15e025
      guybe7 authored
      There's a race between testing DBSIZE and the thread starting.
      If the thread hadn't started by the time we checked DBISZE, no
      keys will have been evicted.
      The correct way is to check the evicted_keys stat.
      0f15e025
  10. 26 Dec, 2021 3 commits
    • Binbin's avatar
      santize dump payload: fix carsh when zset with NAN score (#10002) · e84ccc3f
      Binbin authored
      `zslInsert` with a NAN score will crash the server.
      This one found by the `corrupt-dump-fuzzer`.
      e84ccc3f
    • Meir Shpilraien (Spielrein)'s avatar
      Add FUNCTION DUMP and RESTORE. (#9938) · 365cbf46
      Meir Shpilraien (Spielrein) authored
      Follow the conclusions to support Functions in redis cluster (#9899)
      
      Added 2 new FUNCTION sub-commands:
      1. `FUNCTION DUMP` - dump a binary payload representation of all the functions.
      2. `FUNCTION RESTORE <PAYLOAD> [FLUSH|APPEND|REPLACE]` - give the binary payload extracted
         using `FUNCTION DUMP`, restore all the functions on the given payload. Restore policy can be given to
         control how to handle existing functions (default is APPEND):
         * FLUSH: delete all existing functions.
         * APPEND: appends the restored functions to the existing functions. On collision, abort.
         * REPLACE: appends the restored functions to the existing functions. On collision,
           replace the old function with the new function.
      
      Modify `redis-cli --cluster add-node` to use `FUNCTION DUMP` to get existing functions from
      one of the nodes in the cluster, and `FUNCTION RESTORE` to load the same set of functions
      to the new node. `redis-cli` will execute this step before sending the `CLUSTER MEET` command
      to the new node. If `FUNCTION DUMP` returns an error, assume the current Redis version do not
      support functions and skip `FUNCTION RESTORE`. If `FUNCTION RESTORE` fails, abort and do not send
      the `CLUSTER MEET` command. If the new node already contains functions (before the `FUNCTION RESTORE`
      is sent), abort and do not add the node to the cluster. Test was added to verify
      `redis-cli --cluster add-node` works as expected. 
      365cbf46
    • Meir Shpilraien (Spielrein)'s avatar
      Changed fuction name to be case insensitive. (#9984) · 08ff606b
      Meir Shpilraien (Spielrein) authored
      Use case insensitive string comparison for function names (like we do for commands and configs)
      In addition, add verification that the functions only use the following characters: [a-zA-Z0-9_]
      08ff606b
  11. 23 Dec, 2021 1 commit
    • Yuta Hongo's avatar
      redis-cli: Add OUTPUT_JSON format type (#9954) · 63f606d3
      Yuta Hongo authored
      Introduce `redis-cli --json` option.
      CSV doesn't support Map type, then parsing SLOWLOG or HMSET with multiple args are not helpful.
      
      By default the `--json` implies RESP3, which makes it much more useful, and a `-2`
      option was added to force RESP2.
      When `HELLO 3` fails, it prints a warning message (which can be silenced with `-2`).
      If a user passed `-3` explicitly, the non-interactive mode will also exit with error without
      running the command, while in interactive session it'll keep running after printing the warning.
      
      JSON output would be helpful to parse Redis replies with other tools like jq.
      
      ```
      redis-cli --json slowlog get | jq
      
      [
        [
          1,
          1639677545,
          322362,
          [
            "HMSET",
            "dummy-key",
            "field1",
            "123,456,789... (152 more bytes)",
            "field2",
            "111,222,333... (140 more bytes)",
            "field3",
            "... (349 more arguments)"
          ],
          "127.0.0.1:49312",
          ""
        ]
      ]
      
      ```
      63f606d3
  12. 22 Dec, 2021 6 commits
    • guybe7's avatar
      Sort out mess around propagation and MULTI/EXEC (#9890) · 7ac21307
      guybe7 authored
      The mess:
      Some parts use alsoPropagate for late propagation, others using an immediate one (propagate()),
      causing edge cases, ugly/hacky code, and the tendency for bugs
      
      The basic idea is that all commands are propagated via alsoPropagate (i.e. added to a list) and the
      top-most call() is responsible for going over that list and actually propagating them (and wrapping
      them in MULTI/EXEC if there's more than one command). This is done in the new function,
      propagatePendingCommands.
      
      Callers to propagatePendingCommands:
      1. top-most call() (we want all nested call()s to add to the also_propagate array and just the top-most
         one to propagate them) - via `afterCommand`
      2. handleClientsBlockedOnKeys: it is out of call() context and it may propagate stuff - via `afterCommand`. 
      3. handleClientsBlockedOnKeys edge case: if the looked-up key is already expired, we will propagate the
         expire but will not unblock any client so `afterCommand` isn't called. in that case, we have to propagate
         the deletion explicitly.
      4. cron stuff: active-expire and eviction may also propagate stuff
      5. modules: the module API allows to propagate stuff from just about anywhere (timers, keyspace notifications,
         threads). I could have tried to catch all the out-of-call-context places but it seemed easier to handle it in one
         place: when we free the context. in the spirit of what was done in call(), only the top-most freeing of a module
         context may cause propagation.
      6. modules: when using a thread-safe ctx it's not clear when/if the ctx will be freed. we do know that the module
         must lock the GIL before calling RM_Replicate/RM_Call so we propagate the pending commands when
         releasing the GIL.
      
      A "known limitation", which were actually a bug, was fixed because of this commit (see propagate.tcl):
         When using a mix of RM_Call with `!` and RM_Replicate, the command would propagate out-of-order:
         first all the commands from RM_Call, and then the ones from RM_Replicate
      
      Another thing worth mentioning is that if, in the past, a client would issue a MULTI/EXEC with just one
      write command the server would blindly propagate the MULTI/EXEC too, even though it's redundant.
      not anymore.
      
      This commit renames propagate() to propagateNow() in order to cause conflicts in pending PRs.
      propagatePendingCommands is the only caller of propagateNow, which is now a static, internal helper function.
      
      Optimizations:
      1. alsoPropagate will not add stuff to also_propagate if there's no AOF and replicas
      2. alsoPropagate reallocs also_propagagte exponentially, to save calls to memmove
      
      Bugfixes:
      1. CONFIG SET can create evictions, sending notifications which can cause to dirty++ with modules.
         we need to prevent it from propagating to AOF/replicas
      2. We need to set current_client in RM_Call. buggy scenario:
         - CONFIG SET maxmemory, eviction notifications, module hook calls RM_Call
         - assertion in lookupKey crashes, because current_client has CONFIG SET, which isn't CMD_WRITE
      3. minor: in eviction, call propagateDeletion after notification, like active-expire and all commands
         (we always send a notification before propagating the command)
      7ac21307
    • Oran Agra's avatar
      resolve replication test timing sensitivity - 2nd attempt (#9988) · b7567394
      Oran Agra authored
      issue started failing after #9878 was merged (made an exiting test more sensitive)
      looks like #9982 didn't help, tested this one and it seems to work better.
      
      this commit does two things:
      1. reduce the extra delay i added earlier and instead add more keys, the effect no duration
         of replication is the same, but the intervals in which the server is responsive to the tcl client is higher.
      2. improve the test infra to print context when assert_error fails.
      b7567394
    • Oran Agra's avatar
      resolve replication test timing sensitivity (#9982) · e33e0295
      Oran Agra authored
      issue started failing after #9878 was merged (made an exiting test more sensitive)
      e33e0295
    • Oran Agra's avatar
      Allow most CONFIG SET during loading, block some commands in async-loading (#9878) · 41e6e05d
      Oran Agra authored
      ## background
      Till now CONFIG SET was blocked during loading.
      (In the not so distant past, GET was disallowed too)
      
      We recently (not released yet) added an async-loading mode, see #9323,
      and during that time it'll serve CONFIG SET and any other command.
      And now we realized (#9770) that some configs, and commands are dangerous
      during async-loading.
      
      ## changes
      * Allow most CONFIG SET during loading (both on async-loading and normal loading)
      * Allow CONFIG REWRITE and CONFIG RESETSTAT during loading
      * Block a few config during loading (`appendonly`, `repl-diskless-load`, and `dir`)
      * Block a few commands during loading (list below)
      
      ## the blocked commands:
      * SAVE - obviously we don't wanna start a foregreound save during loading 8-)
      * BGSAVE - we don't mind to schedule one, but we don't wanna fork now
      * BGREWRITEAOF - we don't mind to schedule one, but we don't wanna fork now
      * MODULE - we obviously don't wanna unload a module during replication / rdb loading
        (MODULE HELP and MODULE LIST are not blocked)
      * SYNC / PSYNC - we're in the middle of RDB loading from master, must not allow sync
        requests now.
      * REPLICAOF / SLAVEOF - we're in the middle of replicating, maybe it makes sense to let
        the user abort it, but he couldn't do that so far, i don't wanna take any risk of bugs due to odd state.
      * CLUSTER - only allow [HELP, SLOTS, NODES, INFO, MYID, LINKS, KEYSLOT, COUNTKEYSINSLOT,
        GETKEYSINSLOT, RESET, REPLICAS, COUNT_FAILURE_REPORTS], for others, preserve the status quo
      
      ## other fixes
      * processEventsWhileBlocked had an issue when being nested, this could happen with a busy script
        during async loading (new), but also in a busy script during AOF loading (old). this lead to a crash in
        the scenario described in #6988
      41e6e05d
    • zhugezy's avatar
      Shorten timeouts of CLIENT PAUSE to avoid hanging when tests fail. (#9975) · ad55fbaa
      zhugezy authored
      If a test fails at `wait_for_blocked_clients_count` after the `PAUSE` command,
      It won't send `UNPAUSE` to server, leading to the server hanging until timeout,
      which is bad and hard to debug sometimes when developing.
      This PR tries to fix this.
      
      Timeout in `CLIENT PAUSE` shortened from 1e5 seconds(extremely long) to 50~100 seconds.
      ad55fbaa
    • Hongcai Ren's avatar
      There is mismach between function sdssplitlen() comments and implementation (#4909) · b28dbef5
      Hongcai Ren authored
      when count is 0, return NULL
      b28dbef5
  13. 21 Dec, 2021 1 commit
    • Meir Shpilraien (Spielrein)'s avatar
      Change FUNCTION CREATE, DELETE and FLUSH to be WRITE commands instead of MAY_REPLICATE. (#9953) · 3bcf1084
      Meir Shpilraien (Spielrein) authored
      The issue with MAY_REPLICATE is that all automatic mechanisms to handle
      write commands will not work. This require have a special treatment for:
      * Not allow those commands to be executed on RO replica.
      * Allow those commands to be executed on RO replica from primary connection.
      * Allow those commands to be executed on the RO replica from AOF.
      
      By setting those commands as WRITE commands we are getting all those properties from Redis.
      Test was added to verify that those properties work as expected.
      
      In addition, rearrange when and where functions are flushed. Before this PR functions were
      flushed manually on `rdbLoadRio` and cleaned manually on failure. This contradicts the
      assumptions that functions are data and need to be created/deleted alongside with the
      data. A side effect of this, for example, `debug reload noflush` did not flush the data but
      did flush the functions, `debug loadaof` flush the data but not the functions.
      This PR move functions deletion into `emptyDb`. `emptyDb` (renamed to `emptyData`) will
      now accept an additional flag, `NOFUNCTIONS` which specifically indicate that we do not
      want to flush the functions (on all other cases, functions will be flushed). Used the new flag
      on FLUSHALL and FLUSHDB only! Tests were added to `debug reload` and `debug loadaof`
      to verify that functions behave the same as the data.
      
      Notice that because now functions will be deleted along side with the data we can not allow
      `CLUSTER RESET` to be called from within a function (it will cause the function to be released
      while running), this PR adds `NO_SCRIPT` flag to `CLUSTER RESET`  so it will not be possible
      to be called from within a function. The other cluster commands are allowed from within a
      function (there are use-cases that uses `GETKEYSINSLOT` to iterate over all the keys on a
      given slot). Tests was added to verify `CLUSTER RESET` is denied from within a script.
      
      Another small change on this PR is that `RDBFLAGS_ALLOW_DUP` is also applicable on functions.
      When loading functions, if this flag is set, we will replace old functions with new ones on collisions. 
      3bcf1084