1. 11 Apr, 2023 1 commit
  2. 10 Apr, 2023 2 commits
  3. 09 Apr, 2023 1 commit
    • Ozan Tezcan's avatar
      Add RM_RdbLoad and RM_RdbSave module API functions (#11852) · e55568ed
      Ozan Tezcan authored
      Add `RM_RdbLoad()` and `RM_RdbSave()` to load/save RDB files from the module API. 
      
      In our use case, we have our clustering implementation as a module. As part of this
      implementation, the module needs to trigger RDB save operation at specific points.
      Also, this module delivers RDB files to other nodes (not using Redis' replication).
      When a node receives an RDB file, it should be able to load the RDB. Currently,
      there is no module API to save/load RDB files. 
      
      
      This PR adds four new APIs:
      ```c
      RedisModuleRdbStream *RM_RdbStreamCreateFromFile(const char *filename);
      void RM_RdbStreamFree(RedisModuleRdbStream *stream);
      
      int RM_RdbLoad(RedisModuleCtx *ctx, RedisModuleRdbStream *stream, int flags);
      int RM_RdbSave(RedisModuleCtx *ctx, RedisModuleRdbStream *stream, int flags);
      ```
      
      The first step is to create a `RedisModuleRdbStream` object. This PR provides a function to
      create RedisModuleRdbStream from the filename. (You can load/save RDB with the filename).
      In the future, this API can be extended if needed: 
      e.g., `RM_RdbStreamCreateFromFd()`, `RM_RdbStreamCreateFromSocket()` to save/load
      RDB from an `fd` or a `socket`. 
      
      
      Usage:
      ```c
      /* Save RDB */
      RedisModuleRdbStream *stream = RedisModule_RdbStreamCreateFromFile("example.rdb");
      RedisModule_RdbSave(ctx, stream, 0);
      RedisModule_RdbStreamFree(stream);
      
      /* Load RDB */
      RedisModuleRdbStream *stream = RedisModule_RdbStreamCreateFromFile("example.rdb");
      RedisModule_RdbLoad(ctx, stream, 0);
      RedisModule_RdbStreamFree(stream);
      ```
      e55568ed
  4. 06 Apr, 2023 7 commits
  5. 05 Apr, 2023 1 commit
    • Oran Agra's avatar
      Increase threshold for flaky cache reclaim test (#12004) · f263b6da
      Oran Agra authored
      This test produces 1GB of data and moves it around, and was expecting less
      than 500kb to be present in the system page cache.
      It sometimes fails with up to some 6mb in the page cache (0 in the actual RDB files),
      increasing the threshold. It looks like some background tasks in the container are
      occupying the page cache.
      
      It is safe to ignore the above since we also explicitly check the pages of our dump.rdb
      are not cached (matching `vmtouch -v` to `0%`).
      An additional fix is to match ` 0%` (add space), so that we don't successfully match `10%`.
      
      details in https://github.com/redis/redis/pull/11818
      f263b6da
  6. 04 Apr, 2023 5 commits
    • bodong.ybd's avatar
      Add help message for client setinfo (#11995) · ccc86a91
      bodong.ybd authored
      
      
      The new sub-command was missing from CLIENT HELP
      Co-authored-by: default avatarBinbin <binloveplay1314@qq.com>
      ccc86a91
    • Thomas Fline's avatar
      Document syslog directives in sentinel.conf. (#11889) · 219e85ff
      Thomas Fline authored
      Redis supports syslog integration via these directives, documented in redis.conf.
      While these directives are not documented in sentinel.conf, they do work with Redis-Sentinel.
      It took me a while to realize this, adding them to make it clear.
      219e85ff
    • Subhi Al Hasan's avatar
      check for known-slave in sentinel rewrite config (#11775) · 74b29985
      Subhi Al Hasan authored
      Fix the following config file error
      
      ```
      *** FATAL CONFIG FILE ERROR (Redis 6.2.7) ***
      Reading the configuration file, at line 152
      >>> 'sentinel known-replica XXXX 127.0.0.1 5001'
      Duplicate hostname and port for replica.
      ```
      
      
      that is happening when a user uses the legacy key "known-slave" in
      the config file and a config rewrite occurs. The config rewrite logic won't
      replace the old  line "sentinel known-slave XXXX 127.0.0.1 5001" and
      would add a new line with "sentinel known-replica XXXX 127.0.0.1 5001"
      which results in the error above "Duplicate hostname and port for replica."
      
      example:
      
      Current sentinal.conf
      ```
      ...
      
      sentinel known-slave XXXX 127.0.0.1 5001
      sentinel example-random-option X
      ...
      ```
      after the config rewrite logic runs:
      ```
      ....
      sentinel known-slave XXXX 127.0.0.1 5001
      sentinel example-random-option X
      
      # Generated by CONFIG REWRITE
      sentinel known-replica XXXX 127.0.0.1 5001
      ```
      
      This bug only exists in Redis versions >=6.2 because prior to that it was hidden
      by the effects of this bug https://github.com/redis/redis/issues/5388 that was fixed
      in https://github.com/redis/redis/pull/8271 and was released in versions >=6.2
      74b29985
    • gx's avatar
      Fix local clients detection (#11664) · e1da7241
      gx authored
      Match 127.0.0.0/8 instead of just `127.0.0.1` to detect the local clients.
      e1da7241
    • Binbin's avatar
      Changed activeExpireCycle server.masterhost check to iAmMaster in beforeSleep (#11997) · aee8d1ff
      Binbin authored
      In cluster mode, when a node restart as a replica, it doesn't immediately
      sync with the master, replication is enabled in clusterCron. It means that
      sometime server.masterhost is NULL and we wrongly judge it in beforeSleep.
      
      In this case, we may trigger a fast activeExpireCycle in beforeSleep, but the
      node's flag is actually a replica, that can lead to data inconsistency.  In this
      PR, we use iAmMaster to replace the `server.masterhost == NULL`
      
      This is an overlook in #7001, and more discussion in #11783.
      aee8d1ff
  7. 02 Apr, 2023 2 commits
    • Wen Hui's avatar
      redis-cli - handle sensitive command redaction for variadic CONFIG SET (#11975) · a4a0eab5
      Wen Hui authored
      In the Redis 7.0 and newer version,
      config set command support multiply `<parameter> <value>` pairs, thus the previous
      sensitive command condition does not apply anymore
      
      For example:
      
      The command:
      **config set maxmemory 1GB masteruser aa** will be written to redis_cli historyfile
      
      In this PR, we update the condition for these sensitive commands
      config set masteruser <username>
      config set masterauth <master-password>
      config set requirepass foobared
      a4a0eab5
    • Slava Koyfman's avatar
      Disconnect pub-sub subscribers when revoking `allchannels` permission (#11992) · f38aa6bf
      Slava Koyfman authored
      The existing logic for killing pub-sub clients did not handle the `allchannels`
      permission correctly. For example, if you:
      
          ACL SETUSER foo allchannels
      
      Have a client authenticate as the user `foo` and subscribe to a channel, and then:
      
          ACL SETUSER foo resetchannels
      
      The subscribed client would not be disconnected, though new clients under that user
      would be blocked from subscribing to any channels.
      
      This was caused by an incomplete optimization in `ACLKillPubsubClientsIfNeeded`
      checking whether the new channel permissions were a strict superset of the old ones.
      f38aa6bf
  8. 30 Mar, 2023 2 commits
    • Jason Elbaum's avatar
      Reimplement cli hints based on command arg docs (#10515) · 1f76bb17
      Jason Elbaum authored
      
      
      Now that the command argument specs are available at runtime (#9656), this PR addresses
      #8084 by implementing a complete solution for command-line hinting in `redis-cli`.
      
      It correctly handles nearly every case in Redis's complex command argument definitions, including
      `BLOCK` and `ONEOF` arguments, reordering of optional arguments, and repeated arguments
      (even when followed by mandatory arguments). It also validates numerically-typed arguments.
      It may not correctly handle all possible combinations of those, but overall it is quite robust.
      
      Arguments are only matched after the space bar is typed, so partial word matching is not
      supported - that proved to be more confusing than helpful. When the user's current input
      cannot be matched against the argument specs, hinting is disabled.
      
      Partial support has been implemented for legacy (pre-7.0) servers that do not support
      `COMMAND DOCS`, by falling back to a statically-compiled command argument table.
      On startup, if the server does not support `COMMAND DOCS`, `redis-cli` will now issue
      an `INFO SERVER` command to retrieve the server version (unless `HELLO` has already
      been sent, in which case the server version will be extracted from the reply to `HELLO`).
      The server version will be used to filter the commands and arguments in the command table,
      removing those not supported by that version of the server. However, the static table only
      includes core Redis commands, so with a legacy server hinting will not be supported for
      module commands. The auto generated help.h and the scripts that generates it are gone.
      
      Command and argument tables for the server and CLI use different structs, due primarily
      to the need to support different runtime data. In order to generate code for both, macros
      have been added to `commands.def` (previously `commands.c`) to make it possible to
      configure the code generation differently for different use cases (one linked with redis-server,
      and one with redis-cli).
      
      Also adding a basic testing framework for the command hints based on new (undocumented)
      command line options to `redis-cli`: `--test_hint 'INPUT'` prints out the command-line hint for
      a given input string, and `--test_hint_file <filename>` runs a suite of test cases for the hinting
      mechanism. The test suite is in `tests/assets/test_cli_hint_suite.txt`, and it is run from
      `tests/integration/redis-cli.tcl`.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      Co-authored-by: default avatarViktor Söderqvist <viktor.soderqvist@est.tech>
      1f76bb17
    • Madelyn Olson's avatar
      Fixed tracking of command duration for multi/eval/module/wait (#11970) · 971b177f
      Madelyn Olson authored
      In #11012, we changed the way command durations were computed to handle the same command being executed multiple times. This commit fixes some misses from that commit.
      
      * Wait commands were not correctly reporting their duration if the timeout was reached.
      * Multi/scripts/and modules with RM_Call were not properly resetting the duration between inner calls, leading to them reporting cumulative duration.
      * When a blocked client is freed, the call and duration are always discarded.
      
      This commit also adds an assert if the duration is not properly reset, potentially indicating that a report to call statistics was missed. The assert potentially be removed in the future, as it's mainly intended to detect misses in tests.
      971b177f
  9. 29 Mar, 2023 2 commits
    • Itamar Haber's avatar
      Overhauls command summaries and man pages. (#11942) · 0c3b8b7e
      Itamar Haber authored
      This is an attempt to normalize/formalize command summaries.
      
      Main actions performed:
      
      * Starts with the continuation of the phrase "The XXXX command, when called, ..." for user commands.
      * Starts with "An internal command...", "A container command...", etc... when applicable.
      * Always uses periods.
      * Refrains from referring to other commands. If this is needed, backquotes should be used for command names.
      * Tries to be very clear about the data type when applicable.
      * Tries to mention additional effects, e.g. "The key is created if it doesn't exist" and "The set is deleted if the last member is removed."
      * Prefers being terse over verbose.
      * Tries to be consistent.
      0c3b8b7e
    • Binbin's avatar
      Fix fork done handler wrongly update fsync metrics and enhance AOF_ FSYNC_ALWAYS (#11973) · cb171786
      Binbin authored
      This PR fix several unrelated bugs that were discovered by the same set of tests
      (WAITAOF tests in #11713), could make the `WAITAOF` test hang. 
      
      The change in `backgroundRewriteDoneHandler` is about MP-AOF.
      That leftover / old code assumes that we started a new AOF file just now
      (when we have a new base into which we're gonna incrementally write), but
      the fact is that with MP-AOF, the fork done handler doesn't really affect the
      incremental file being maintained by the parent process, there's no reason to
      re-issue `SELECT`, and no reason to update any of the fsync variables in that flow.
      This should have been deleted with MP-AOF (introduced in #9788, 7.0).
      The damage is that the update to `aof_fsync_offset` will cause us to miss an fsync
      in `flushAppendOnlyFile`, that happens if we stop write commands in `AOF_FSYNC_EVERYSEC`
      while an AOFRW is in progress. This caused a new `WAITAOF` test to sometime hang forever.
      
      Also because of MP-AOF, we needed to change `aof_fsync_offset` to `aof_last_incr_fsync_offset`
      and match it to `aof_last_incr_size` in `flushAppendOnlyFile`. This is because in the past we compared
      `aof_fsync_offset` and `aof_current_size`, but with MP-AOF it could be the total AOF file will be
      smaller after AOFRW, and the (already existing) incr file still has data that needs to be fsynced.
      
      The change in `flushAppendOnlyFile`, about the `AOF_FSYNC_ALWAYS`, it is follow #6053
      (the details is in #5985), we also check `AOF_FSYNC_ALWAYS` to handle a case where
      appendfsync is changed from everysec to always while there is data that's written but not yet fsynced.
      cb171786
  10. 28 Mar, 2023 3 commits
  11. 27 Mar, 2023 1 commit
    • Rafi Einstein's avatar
      Clang: fix for -flto argument (#11961) · 557ca05d
      Rafi Einstein authored
      Starting with the recent #11926 Makefile specifies `-flto=auto` which is unsupported on clang.
      Additionally, detecting clang correctly requires actually running it, since on MacOS gcc can be an alias for clang.
      557ca05d
  12. 26 Mar, 2023 2 commits
    • Binbin's avatar
      Fix redis-cli cluster test timing issue (#11887) · aa2403ca
      Binbin authored
      This test fails sporadically:
      ```
      *** [err]: Migrate the last slot away from a node using redis-cli in tests/unit/cluster/cli.tcl
      cluster size did not reach a consistent size 4
      ```
      
      I guess the time (5s) of wait_for_cluster_size is not enough,
      usually, the waiting time for our other tests for cluster
      consistency is 50s, so also changing it to 50s.
      aa2403ca
    • Binbin's avatar
      Add COMMAND COUNT test to cover reply-schemas-validator test (#11971) · 2cc99c69
      Binbin authored
      Since we remove the COMMAND COUNT call in sentinel test in #11950,
      reply-schemas-validator started reporting this error:
      ```
      WARNING! The following commands were not hit at all:
        command|count
        ERROR! at least one command was not hit by the tests
      ```
      
      This PR add a COMMAND COUNT test to cover it and also fix some
      typos in req-res-log-validator.py
      2cc99c69
  13. 24 Mar, 2023 2 commits
  14. 23 Mar, 2023 3 commits
  15. 22 Mar, 2023 6 commits