- 04 Jan, 2022 2 commits
-
-
yoav-steinberg authored
This makes redis-cli --replica much faster and reduces COW/fork risks on server side. This commit also improves the RDB filtering via REPLCONF rdb-filter-only to support no "include" specifiers at all.
-
Binbin authored
In monitor/pubsub mode, if the server closes the connection, for example, use `CLIENT KILL`, redis-cli will exit directly without printing any error messages. This commit ensures that redis-cli will try to print the error messages before exiting. Also there is a minor cleanup for restart, see the example below. before: ``` 127.0.0.1:6379> monitor OK [root@ redis]# 127.0.0.1:6379> subscribe channel Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "channel" 3) (integer) 1 [root@ redis]# 127.0.0.1:6379> restart 127.0.0.1:6379> get keyUse 'restart' only in Lua debugging mode. (nil) ``` after: ``` 127.0.0.1:6379> monitor OK Error: Server closed the connection [root@ redis]# 127.0.0.1:6379> subscribe channel Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "channel" 3) (integer) 1 Error: Server closed the connection [root@ redis]# 127.0.0.1:6379> restart Use 'restart' only in Lua debugging mode. ```
-
- 03 Jan, 2022 1 commit
-
-
Harkrishn Patro authored
This commit implements a sharded pubsub implementation based off of shard channels. Co-authored-by:
Harkrishn Patro <harkrisp@amazon.com> Co-authored-by:
Madelyn Olson <madelyneolson@gmail.com>
-
- 02 Jan, 2022 1 commit
-
-
yoav-steinberg authored
This is needed in order to ease the deployment of functions for ephemeral cases, where user needs to spin up a server with functions pre-loaded. #### Details: * Added `--functions-rdb` option to _redis-cli_. * Functions only rdb via `REPLCONF rdb-filter-only functions`. This is a placeholder for a space separated inclusion filter for the RDB. In the future can be `REPLCONF rdb-filter-only "functions db:3 key-patten:user*"` and a complementing `rdb-filter-exclude` `REPLCONF` can also be added. * Handle "slave requirements" specification to RDB saving code so we can use the same RDB when different slaves express the same requirements (like functions-only) and not share the RDB when their requirements differ. This is currently just a flags `int`, but can be extended to a more complex structure with various filter fields. * make sure to support filters only in diskless replication mode (not to override the persistence file), we do that by forcing diskless (even if disabled by config) other changes: * some refactoring in rdb.c (extract portion of a big function to a sub-function) * rdb_key_save_delay used in AOFRW too * sendChildInfo takes the number of updated keys (incremental, rather than absolute) Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 30 Dec, 2021 1 commit
-
-
Binbin authored
There are two changes in this commit: 1. Add -X option to redis-cli. Currently `-x` can only be used to provide the last argument, so you can do `redis-cli dump keyname > key.dump`, and then do `redis-cli -x restore keyname 0 < key.dump`. But what if you want to add the replace argument (which comes last?). oran suggested adding such usage: `redis-cli -X <tag> restore keyname <tag> replace < key.dump` i.e. you're able to provide a string in the arguments that's gonna be substituted with the content from stdin. Note that the tag name should not conflict with others non-replaced args. And the -x and -X options are conflicting. Some usages: ``` [root]# echo mypasswd | src/redis-cli -X passwd_tag mset username myname password passwd_tag OK [root]# echo username > username.txt [root]# head -c -1 username.txt | src/redis-cli -X name_tag mget name_tag password 1) "myname" 2) "mypasswd\n" ``` 2. Handle the combination of both `-x` and `--cluster` or `-X` and `--cluster` Extend the broadcast option to receive the last arg or <tag> arg from the stdin. Now we can use `redis-cli -x --cluster call <host>:<port> cmd`, or `redis-cli -X <tag> --cluster call <host>:<port> cmd <tag>`. (support part of #9899)
-
- 26 Dec, 2021 1 commit
-
-
Meir Shpilraien (Spielrein) authored
Follow the conclusions to support Functions in redis cluster (#9899) Added 2 new FUNCTION sub-commands: 1. `FUNCTION DUMP` - dump a binary payload representation of all the functions. 2. `FUNCTION RESTORE <PAYLOAD> [FLUSH|APPEND|REPLACE]` - give the binary payload extracted using `FUNCTION DUMP`, restore all the functions on the given payload. Restore policy can be given to control how to handle existing functions (default is APPEND): * FLUSH: delete all existing functions. * APPEND: appends the restored functions to the existing functions. On collision, abort. * REPLACE: appends the restored functions to the existing functions. On collision, replace the old function with the new function. Modify `redis-cli --cluster add-node` to use `FUNCTION DUMP` to get existing functions from one of the nodes in the cluster, and `FUNCTION RESTORE` to load the same set of functions to the new node. `redis-cli` will execute this step before sending the `CLUSTER MEET` command to the new node. If `FUNCTION DUMP` returns an error, assume the current Redis version do not support functions and skip `FUNCTION RESTORE`. If `FUNCTION RESTORE` fails, abort and do not send the `CLUSTER MEET` command. If the new node already contains functions (before the `FUNCTION RESTORE` is sent), abort and do not add the node to the cluster. Test was added to verify `redis-cli --cluster add-node` works as expected.
-
- 23 Dec, 2021 1 commit
-
-
Yuta Hongo authored
Introduce `redis-cli --json` option. CSV doesn't support Map type, then parsing SLOWLOG or HMSET with multiple args are not helpful. By default the `--json` implies RESP3, which makes it much more useful, and a `-2` option was added to force RESP2. When `HELLO 3` fails, it prints a warning message (which can be silenced with `-2`). If a user passed `-3` explicitly, the non-interactive mode will also exit with error without running the command, while in interactive session it'll keep running after printing the warning. JSON output would be helpful to parse Redis replies with other tools like jq. ``` redis-cli --json slowlog get | jq [ [ 1, 1639677545, 322362, [ "HMSET", "dummy-key", "field1", "123,456,789... (152 more bytes)", "field2", "111,222,333... (140 more bytes)", "field3", "... (349 more arguments)" ], "127.0.0.1:49312", "" ] ] ```
-
- 19 Dec, 2021 1 commit
-
-
Binbin authored
1. Local variable 's' hides a parameter of the same name. ``` int anetTcpAccept(char *err, int s, char *ip, size_t ip_len, int *port) { if ((fd = anetGenericAccept(err,s,(struct sockaddr*)&sa,&salen)) == ANET_ERR){} } ``` Change the parameter name from `s` to `serversock`, also unified with the header file definition. 2. Comparison is always false because i <= 48. ``` for (i = 0; i < DICT_STATS_VECTLEN-1; i++) { // i < 49 (i == DICT_STATS_VECTLEN-1)?">= ":"", // i == 49 } ``` `i == DICT_STATS_VECTLEN-1` always result false, it is a dead code. 3. Empty block without comment. `else if (!strcasecmp(opt,"ch")) {}`, add a comment to avoid warnings. 4. Bit field fill of type int should have explicitly unsigned integral, explicitly signed integral, or enumeration type. Modify `int fill: QL_FILL_BITS;` to `unsigned int fill: QL_FILL_BITS;` 5. The result of this call to reconnectingRedisCommand is not checked for null, but 80% of calls to reconnectingRedisCommand check for null. Just a cleanup job like others.
-
- 02 Dec, 2021 1 commit
-
-
Binbin authored
When we use monitor in redis-cli but encounter an error reply, we will get stuck until we press Ctrl-C to quit. This is a harmless bug. It might be useful if we add parameters to monitor in the future, suck as monitoring only selected db. before: ``` 127.0.0.1:6379> monitor wrong (error) ERR wrong number of arguments for 'monitor' command or subcommand ^C(9.98s) 127.0.0.1:6379> ``` after: ``` 127.0.0.1:6379> monitor wrong (error) ERR wrong number of arguments for 'monitor' command or subcommand 127.0.0.1:6379> ```
-
- 13 Oct, 2021 1 commit
-
-
Ofir Luzon authored
Adding -i option (sleep interval) of repeat and bigkeys to redis-cli --scan. When the keyspace contains many already expired keys scanning the dataset with redis-cli --scan can impact the performance Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 12 Oct, 2021 1 commit
-
-
Ning Xie authored
bigkeys sleep is defined each 100 scanned keys, and it is checked it only between scan cycles. In cases that scan does not return exactly 10 keys it will never sleep. In addition the comment was sleep each 100 SCANs but it was 100 scanned keys.
-
- 23 Sep, 2021 1 commit
-
-
yoav-steinberg authored
### Description A mechanism for disconnecting clients when the sum of all connected clients is above a configured limit. This prevents eviction or OOM caused by accumulated used memory between all clients. It's a complimentary mechanism to the `client-output-buffer-limit` mechanism which takes into account not only a single client and not only output buffers but rather all memory used by all clients. #### Design The general design is as following: * We track memory usage of each client, taking into account all memory used by the client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date after reading from the socket, after processing commands and after writing to the socket. * Based on the used memory we sort all clients into buckets. Each bucket contains all clients using up up to x2 memory of the clients in the bucket below it. For example up to 1m clients, up to 2m clients, up to 4m clients, ... * Before processing a command and before sleep we check if we're over the configured limit. If we are we start disconnecting clients from larger buckets downwards until we're under the limit. #### Config `maxmemory-clients` max memory all clients are allowed to consume, above this threshold we disconnect clients. This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%` would mean 10% of `maxmemory`). #### Important code changes * During the development I encountered yet more situations where our io-threads access global vars. And needed to fix them. I also had to handle keeps the clients sorted into the memory buckets (which are global) while their memory usage changes in the io-thread. To achieve this I decided to simplify how we check if we're in an io-thread and make it much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking if the client is in an io-thread (it wasn't used for anything else) and just used the global `io_threads_op` variable the same way to check during writes. * I optimized the cleanup of the client from the `clients_pending_read` list on client freeing. We now store a pointer in the `client` struct to this list so we don't need to search in it (`pending_read_list_node`). * Added `evicted_clients` stat to `INFO` command. * Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the client eviction mechanism. Added corrosponding 'e' flag in the client info string. * Added `multi-mem` field in the client info string to show how much memory is used up by buffered multi commands. * Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and channels (partially), tracking prefixes (partially). * CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so clients will be disconnected between processing different clients and not only before sleep. This new function can be used in the future for work we want to do outside the command processing loop but don't want to wait for all clients to be processed before we get to it. Specifically I wanted to handle output-buffer-limit related closing before we process client eviction in case the two race with each other. * Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction buckets. * Each client now holds a pointer to the client eviction memory usage bucket it belongs to and listNode to itself in that bucket for quick removal. * Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value indicating no io-threading is currently being executed. * In order to track memory used by each clients in real-time we can't rely on updating these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()` (used to be `clientsCronTrackClientsMemUsage()`) after command processing, after writing data to pubsub clients, after writing the output buffer and after reading from the socket (and maybe other places too). The function is written to be fast. * Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before processing a command (before performing oom-checks and key-eviction). * All clients memory usage buckets are grouped as follows: * All clients using less than 64k. * 64K..128K * 128K..256K * ... * 2G..4G * All clients using 4g and up. * Added client-eviction.tcl with a bunch of tests for the new mechanism. * Extended maxmemory.tcl to test the interaction between maxmemory and maxmemory-clients settings. * Added an option to flag a numeric configuration variable as a "percent", this means that if we encounter a '%' after the number in the config file (or config set command) we consider it as valid. Such a number is store internally as a negative value. This way an integer value can be interpreted as either a percent (negative) or absolute value (positive). This is useful for example if some numeric configuration can optionally be set to a percentage of something else. Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 14 Sep, 2021 1 commit
-
-
filipe oliveira authored
- Add `-u <uri>` command line option to support `redis://` URI scheme. - included server connection information object (`struct cliConnInfo`), used to describe an ip:port pair, db num user input, and user:pass to avoid a large number of function arguments. - Using sds on connection info strings for redis-benchmark/redis-cli Co-authored-by:
yoav-steinberg <yoav@monfort.co.il>
-
- 22 Aug, 2021 1 commit
-
-
Viktor Söderqvist authored
Also make sure function can't return NULL by another assert.
-
- 10 Aug, 2021 1 commit
-
-
Huang Zhw authored
Abort cli blocking modes with SIGINT without exiting the cli. Co-authored-by:
charsyam <charsyam@gmail.com>
-
- 05 Aug, 2021 1 commit
-
-
yoav-steinberg authored
Reduce dict struct memory overhead on 64bit dict size goes down from jemalloc's 96 byte bin to its 56 byte bin. summary of changes: - Remove `privdata` from callbacks and dict creation. (this affects many files, see "Interface change" below). - Meld `dictht` struct into the `dict` struct to eliminate struct padding. (this affects just dict.c and defrag.c) - Eliminate the `sizemask` field, can be calculated from size when needed. - Convert the `size` field into `size_exp` (exponent), utilizes one byte instead of 8. Interface change: pass dict pointer to dict type call back functions. This is instead of passing the removed privdata field. In the future if we'd like to have private data in the callbacks we can extract it from the dict type. We can extend dictType to include a custom dict struct allocator and use it to allocate more data at the end of the dict struct. This data can then be used to store private data later acccessed by the callbacks.
-
- 03 Aug, 2021 1 commit
-
-
filipe oliveira authored
Add the -x option (Read last argument from STDIN) on redis-benchmark. Other changes: To be able to use the code from redis-cli some helper methods were moved to cli_common.(h|c) Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 02 Aug, 2021 1 commit
-
-
Huang Zhw authored
When redis-cli received ASK, it used string matching wrong and didn't handle it. When we access a slot which is in migrating state, it maybe return ASK. After redirect to the new node, we need send ASKING command before retry the command. In this PR after redis-cli receives ASK, we send a ASKING command before send the origin command after reconnecting. Other changes: * Make redis-cli -u and -c (unix socket and cluster mode) incompatible with one another. * When send command fails, we avoid the 2nd reconnect retry and just print the error info. Users will decide how to do next. See #9277. * Add a test faking two redis nodes in TCL to just send ASK and OK in redis protocol to test ASK behavior. Co-authored-by:
Viktor Söderqvist <viktor.soderqvist@est.tech> Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 15 Jul, 2021 1 commit
-
-
ZEEKLING authored
-
- 07 Jul, 2021 1 commit
-
-
Mikhail Fesenko authored
Direct redis-cli repl prints to stderr, because --rdb can print to stdout. fflush stdout after responses (#9136) 1. redis-cli can output --rdb data to stdout but redis-cli also write some messages to stdout which will mess up the rdb. 2. Make redis-cli flush stdout when printing a reply This was needed in order to fix a hung in redis-cli test that uses --replica. Note that printf does flush when there's a newline, but fwrite does not. 3. fix the redis-cli --replica test which used to pass previously because it didn't really care what it read, and because redis-cli used printf to print these other things to stdout. 4. improve redis-cli --replica test to run with both diskless and disk-based. Co-authored-by:
Oran Agra <oran@redislabs.com> Co-authored-by:
Viktor Söderqvist <viktor@zuiderkwast.se>
-
- 05 Jul, 2021 1 commit
-
-
uriyage authored
-
- 30 Jun, 2021 2 commits
-
-
Mikhail Fesenko authored
A change in redis 6.2 caused redis-cli --rdb that's directed to stdout to fail because fsync fails. This commit avoids doing ftruncate (fails with a warning) and fsync (fails with an error) when the output file is `-`, and adds the missing documentation that `-` means stdout. Co-authored-by:
Oran Agra <oran@redislabs.com> Co-authored-by:
Wang Yuan <wangyuancode@163.com>
-
Huang Zhw authored
If user executes redis-cli --help, the content should be output to stdout and exits with 0.
-
- 21 Jun, 2021 1 commit
-
-
Maxim Galushka authored
Fixes #6792. Added support of REDIS_REPLY_SET in raw and csv output of `./redis-cli` Test: run commands to test: ./redis-cli -3 --csv COMMAND ./redis-cli -3 --raw COMMAND Now they are returning resuts, were failing with: "Unknown reply type: 10" before the change.
-
- 10 Jun, 2021 1 commit
-
-
Binbin authored
This PR adds a spell checker CI action that will fail future PRs if they introduce typos and spelling mistakes. This spell checker is based on blacklist of common spelling mistakes, so it will not catch everything, but at least it is also unlikely to cause false positives. Besides that, the PR also fixes many spelling mistakes and types, not all are a result of the spell checker we use. Here's a summary of other changes: 1. Scanned the entire source code and fixes all sorts of typos and spelling mistakes (including missing or extra spaces). 2. Outdated function / variable / argument names in comments 3. Fix outdated keyspace masks error log when we check `config.notify-keyspace-events` in loadServerConfigFromString. 4. Trim the white space at the end of line in `module.c`. Check: https://github.com/redis/redis/pull/7751 5. Some outdated https link URLs. 6. Fix some outdated comment. Such as: - In README: about the rdb, we used to said create a `thread`, change to `process` - dbRandomKey function coment (about the dictGetRandomKey, change to dictGetFairRandomKey) - notifyKeyspaceEvent fucntion comment (add type arg) - Some others minor fix in comment (Most of them are incorrectly quoted by variable names) 7. Modified the error log so that users can easily distinguish between TCP and TLS in `changeBindAddr`
-
- 08 Jun, 2021 2 commits
-
-
ikeberlein authored
redis-cli is grep friendly for all commands but SUBSCRIBE/PSUBSCRIBE. it is unable to process output from these commands line by line piped to another program because of output buffering. to overcome this situation I propose to flush stdout each time when it is written with reply from these commands the same way it is already done for all other commands.
-
Huang Zhw authored
* clusterManagerAddSlots check argv_idx error. * clusterManagerLoadInfoFromNode remove unused param opts. * redis-cli node->ip may be an sds or a c string. Using %s to format is always right, %S may be wrong. * In clusterManagerFixOpenSlot clusterManagerBumpEpoch call is redundant, because it is already called in clusterManagerSetSlotOwner. * redis-cli cluster help add more commands in help messages.
-
- 02 Jun, 2021 1 commit
-
-
Huang Zhw authored
In clusterManagerCommandImport strcat was used to concat COPY and REPLACE, the space maybe not enough. If we use --cluster-replace but not --cluster-copy, the MIGRATE command contained COPY instead of REPLACE.
-
- 19 May, 2021 1 commit
-
-
Huang Zhw authored
which make interleaved_len bigger and may access array out of range.
-
- 18 May, 2021 2 commits
-
-
Huang Zhw authored
Currently in redis-cli only AUTH and ACL SETUSER bypass history file. We add CONFIG SET masterauth/masteruser/requirepass, HELLO with AUTH, MIGRATE with AUTH or AUTH2 to bypass history file too. The drawback is HELLO and MIGRATE's code is a mess. Someday if we change these commands, we have to change here too.
-
Binbin authored
There's an infinite loop when redis-cli fails to connect in cluster mode. This commit adds a 1 second sleep to prevent flooding the console with errors. It also adds a specific error print in a few places that could have error without printing anything. Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 17 May, 2021 1 commit
-
-
Huang Zhw authored
There are two bugs in redis-cli hints: * The hints of commands with subcommands lack first params. * When search matching command of currently input, we should find the command with longest matching prefix. If not COMMAND INFO will always match COMMAND and display no hints.
-
- 06 May, 2021 1 commit
-
-
Huang Zhw authored
when SELECT fails, we should reset dbnum to 0, so the prompt will not display incorrectly. Additionally when SELECT and HELLO fail, we output message to inform it. Add config.input_dbnum which means the dbnum about to select. And config.dbnum means currently selected dbnum. When users succeed to select db, config.dbnum and config.input_dbnum will be the same. When users select db failed, config.input_dbnum will be kept. Next time if users auth success, config.input_dbnum will be automatically selected. When reconnect, we should select the origin dbnum. Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 04 May, 2021 1 commit
-
-
Binbin authored
-
- 03 May, 2021 1 commit
-
-
Oran Agra authored
-
- 29 Apr, 2021 1 commit
-
-
Huang Zhw authored
prefix args not all args. So when we help commands with subcommands, all subcommands will be output.
-
- 28 Apr, 2021 1 commit
-
-
Binbin authored
When redis-cli was used with both -c (cluster) and -s (unix socket), it would have kept trying to use that unix socket, even if it got redirected by the cluster (resulting in an infinite loop).
-
- 26 Apr, 2021 1 commit
-
-
Andy Pan authored
Most of the ae.c backends didn't explicitly handle errors, and instead ignored all errors and did an implicit retry. This is desired for EAGAIN and EINTER, but in case of other systematic errors, we prefer to fail and log the error we got rather than get into a busy loop.
-
- 21 Apr, 2021 1 commit
-
-
Seunghyun Lee authored
-
- 05 Apr, 2021 1 commit
-
-
Huang Zhw authored
When DBSIZE failed (e.g. on AUTH error), the printed error didn't reflect the reason.
-