- 14 Sep, 2021 2 commits
-
-
filipe oliveira authored
- Add `-u <uri>` command line option to support `redis://` URI scheme. - included server connection information object (`struct cliConnInfo`), used to describe an ip:port pair, db num user input, and user:pass to avoid a large number of function arguments. - Using sds on connection info strings for redis-benchmark/redis-cli Co-authored-by:
yoav-steinberg <yoav@monfort.co.il>
-
Viktor Söderqvist authored
List functions operating on elements by index: * RM_ListGet * RM_ListSet * RM_ListInsert * RM_ListDelete Iteration is done using a simple for loop over indices. The index based functions use an internal iterator as an optimization. This is explained in the docs: ``` * Many of the list functions access elements by index. Since a list is in * essence a doubly-linked list, accessing elements by index is generally an * O(N) operation. However, if elements are accessed sequentially or with * indices close together, the functions are optimized to seek the index from * the previous index, rather than seeking from the ends of the list. * * This enables iteration to be done efficiently using a simple for loop: * * long n = RM_ValueLength(key); * for (long i = 0; i < n; i++) { * RedisModuleString *elem = RedisModule_ListGet(key, i); * // Do stuff... * } ```
-
- 13 Sep, 2021 1 commit
-
-
zhaozhao.zz authored
The main idea is how to allow a master to load replication info from RDB file when rebooting, if master can load replication info it means that replicas may have the chance to psync with master, it can save much traffic. The key point is we need guarantee safety and consistency, so there are two differences between master and replica: 1. master would load the replication info as secondary ID and offset, in case other masters have the same replid. 2. when master loading RDB, it would propagate expired keys as DEL command to replication backlog, then replica can receive these commands to delete stale keys. p.s. the expired keys when RDB loading is useful for users, so we show it as `rdb_last_load_keys_expired` and `rdb_last_load_keys_loaded` in info persistence. Moreover, after load replication info, master should update `no_replica_time` in case loading RDB cost too long time.
-
- 12 Sep, 2021 1 commit
-
-
Huang Zhw authored
Make bitpos/bitcount support bit index: ``` BITPOS key bit [start [end [BIT|BYTE]]] BITCOUNT key [start end [BIT|BYTE]] ``` The default behavior is `BYTE`, so these commands are still compatible with old.
-
- 11 Sep, 2021 1 commit
-
-
David CARLIER authored
-
- 10 Sep, 2021 1 commit
-
-
zhaozhao.zz authored
-
- 09 Sep, 2021 8 commits
-
-
sundb authored
Part two of implementing #8702 (zset), after #8887. ## Description of the feature Replaced all uses of ziplist with listpack in t_zset, and optimized some of the code to optimize performance. ## Rdb format changes New `RDB_TYPE_ZSET_LISTPACK` rdb type. ## Rdb loading improvements: 1) Pre-expansion of dict for validation of duplicate data for listpack and ziplist. 2) Simplifying the release of empty key objects when RDB loading. 3) Unify ziplist and listpack data verify methods for zset and hash, and move code to rdb.c. ## Interface changes 1) New `zset-max-listpack-entries` config is an alias for `zset-max-ziplist-entries` (same with `zset-max-listpack-value`). 2) OBJECT ENCODING will return listpack instead of ziplist. ## Listpack improvements: 1) Add `lpDeleteRange` and `lpDeleteRangeWithEntry` functions to delete a range of entries from listpack. 2) Improve the performance of `lpCompare`, converting from string to integer is faster than converting from integer to string. 3) Replace `snprintf` with `ll2string` to improve performance in converting numbers to strings in `lpGet()`. ## Zset improvements: 1) Improve the performance of `zzlFind` method, use `lpFind` instead of `lpCompare` in a loop. 2) Use `lpDeleteRangeWithEntry` instead of `lpDelete` twice to delete a element of zset. ## Tests 1) Add some unittests for `lpDeleteRange` and `lpDeleteRangeWithEntry` function. 2) Add zset RDB loading test. 3) Add benchmark test for `lpCompare` and `ziplsitCompare`. 4) Add empty listpack zset corrupt dump test.
-
Madelyn Olson authored
Throw an error when a user is provided multiple times on the command line instead of silently throwing one of them away. Remove unneeded validation for validating users on ACL load.
-
yancz2000 authored
Add make test-cluster option
-
yvette903 authored
A write request may be paused unexpectedly because `server.client_pause_end_time` is old. **Recreate this:** redis-cli -p 6379 127.0.0.1:6379> client pause 500000000 write OK 127.0.0.1:6379> client unpause OK 127.0.0.1:6379> client pause 10000 write OK 127.0.0.1:6379> set key value The write request `set key value` is paused util the timeout of 500000000 milliseconds was reached. **Fix:** reset `server.client_pause_end_time` = 0 in `unpauseClients`
-
Binbin authored
We want to add COUNT option for BLPOP. But we can't do it without breaking compatibility due to the command arguments syntax. So this commit introduce two new commands. Syntax for the new LMPOP command: `LMPOP numkeys [<key> ...] LEFT|RIGHT [COUNT count]` Syntax for the new BLMPOP command: `BLMPOP timeout numkeys [<key> ...] LEFT|RIGHT [COUNT count]` Some background: - LPOP takes one key, and can return multiple elements. - BLPOP takes multiple keys, but returns one element from just one key. - LMPOP can take multiple keys and return multiple elements from just one key. Note that LMPOP/BLMPOP can take multiple keys, it eventually operates on just one key. And it will propagate as LPOP or RPOP with the COUNT option. As a new command, it still return NIL if we can't pop any elements. For the normal response is nested arrays in RESP2 and RESP3, like: ``` LMPOP/BLMPOP 1) keyname 2) 1) element1 2) element2 ``` I.e. unlike BLPOP that returns a key name and one element so it uses a flat array, and LPOP that returns multiple elements with no key name, and again uses a flat array, this one has to return a nested array, and it does for for both RESP2 and RESP3 (like SCAN does) Some discuss can see: #766 #8824
-
Huang Zhw authored
Add two INFO metrics: ``` total_active_defrag_time:12345 current_active_defrag_time:456 ``` `current_active_defrag_time` if greater than 0, means how much time has passed since active defrag started running. If active defrag stops, this metric is reset to 0. `total_active_defrag_time` means total time the fragmentation was over the defrag threshold since the server started. This is a followup PR for #9031
-
Wang Yuan authored
* Delay to discard cache master when full synchronization * Don't disconnect with replicas before loading transferred RDB when full sync Previously, once replica need to start full synchronization with master, it will discard cached master whatever full synchronization is failed or not. Now we discard cached master only when transferring RDB is finished and start to change data space, this make replica could start partial resynchronization with another new master if new master is failed during full synchronization.
-
chenyang8094 authored
When parsing an array type reply, ctx will be lost when recursively parsing its elements, which will cause a memory leak in automemory mode. This is a result of the changes in #9202 Add test for callReplyParseCollection fix
-
- 08 Sep, 2021 2 commits
-
-
chenyang8094 authored
-
zhaozhao.zz authored
When a replica paused, it would not apply any commands event the command comes from master, if we feed the non-applied command to replication stream, the replication offset would be wrong, and data would be lost after failover(since replica's `master_repl_offset` grows but command is not applied). To fix it, here are the changes: * Don't update replica's replication offset or propagate commands to sub-replicas when it's paused in `commandProcessed`. * Show `slave_read_repl_offset` in info reply. * Add an assert to make sure master client should never be blocked unless pause or module (some modules may use block way to do background (parallel) processing and forward original block module command to the replica, it's not a good way but it can work, so the assert excludes module now, but someday in future all modules should rewrite block command to propagate like what `BLPOP` does).
-
- 06 Sep, 2021 1 commit
-
-
Viktor Söderqvist authored
Until now, giving a negative index seeks from the end of a list and a positive seeks from the beginning. This change makes it seek from the nearest end, regardless of the sign of the given index. quicklistIndex is used by all list commands which operate by index. LINDEX key 999999 in a list if 1M elements is greately optimized by this change. Latency is cut by 75%. LINDEX key -1000000 in a list of 1M elements, likewise. LRANGE key -1 -1 is affected by this, since LRANGE converts the indices to positive numbers before seeking. The tests for corrupt dumps are updated to make sure the corrup data is seeked in the same direction as before.
-
- 02 Sep, 2021 2 commits
-
-
guybe7 authored
1. MIGRATE has a potnetial key arg in argv[3]. It should be reflected in the command table. 2. getKeysUsingCommandTable should never free getKeysResult, it is always freed by the caller) The reason we never encountered this double-free bug is that almost always getKeysResult uses the statis buffer and doesn't allocate a new one.
-
sundb authored
Normally we execute the read event first and then the write event. When the barrier is set, we will do it reverse. However, under `kqueue`, if an `fd` has both read and write events, reading the event using `kevent` will generate two events, which will result in uncontrolled read and write timing. This also means that the guarantees of AOF `appendfsync` = `always` are not met on MacOS without this fix. The main change to this pr is to cache the events already obtained when reading them, so that if the same `fd` occurs again, only the mask in the cache is updated, rather than a new event is generated. This was exposed by the following test failure on MacOS: ``` *** [err]: AOF fsync always barrier issue in tests/integration/aof.tcl Expected 544 != 544 (context: type eval line 26 cmd {assert {$size1 != $size2}} proc ::test) ```
-
- 31 Aug, 2021 1 commit
-
-
Viktor Söderqvist authored
* Enhance dict to support arbitrary metadata carried in dictEntry Co-authored-by:
Viktor Söderqvist <viktor.soderqvist@est.tech> * Rewrite slot-to-keys mapping to linked lists using dict entry metadata This is a memory enhancement for Redis Cluster. The radix tree slots_to_keys (which duplicates all key names prefixed with their slot number) is replaced with a linked list for each slot. The dict entries of the same cluster slot form a linked list and the pointers are stored as metadata in each dict entry of the main DB dict. This commit also moves the slot-to-key API from db.c to cluster.c. Co-authored-by:
Jim Brunner <brunnerj@amazon.com>
-
- 30 Aug, 2021 1 commit
-
-
Wang Yuan authored
We implement incremental data sync in rio.c by call fsync, on slow disk, that may cost a lot of time, sync_file_range could provide async fsync, so we could serialize key/value and sync file data at the same time. > one tip for sync_file_range usage: http://lkml.iu.edu/hypermail/linux/kernel/1005.2/01845.html Additionally, this change avoids a single large write to be used, which can result in a mass of dirty pages in the kernel (increasing the risk of someone else's write to block). On HDD, current solution could reduce approximate half of dumping RDB time, this PR costs 50s for dump 7.7G rdb but unstable branch costs 93s. On NVME SSD, this PR can't reduce much time, this PR costs 40s, unstable branch costs 48s. Moreover, I find calling data sync every 4MB is better than 32MB.
-
- 29 Aug, 2021 2 commits
-
-
Binbin authored
This one follow #9313 and goes deeper (validation of config file parsing) Move the check/update logic to a new updateClientOutputBufferLimit function. So that it can be used in CONFIG SET and config file parsing.
-
Viktor Söderqvist authored
1. The output of --help: * On the Usage line, just write [OPTIONS] [COMMAND ARGS...] instead listing only a few arbitrary options and no command. * For --cluster, describe that if the command is supplied on the command line, the key must contain "{tag}". Otherwise, the command will not be sent to the right cluster node. * For -r, add a note that if -r is omitted, all commands in a benchmark will use the same key. Also align the description. * For -t, describe that -t is ignored if a command is supplied on the command line. 2. Print a warning if -t is present when a specific command is supplied. 3. Print all warnings and errors to stderr. 4. Remove -e from calls in redis-benchmark test suite.
-
- 25 Aug, 2021 1 commit
-
-
Huang Zhw authored
In multipe threads mode, every thread output throughput info. This may cause some problems: - Bug in https://github.com/redis/redis/pull/8615; - The show throughput is called too frequently; - showThroughput which updates shared variable lacks synchronization mechanism. This commit also reverts changes in #8615 and changes time event interval to macro.
-
- 24 Aug, 2021 1 commit
-
-
Garen Chan authored
When `decr_step` is greater than `oldlimit`, the final `bestlimit` may be invalid. For example, oldlimit = 10, decr_step = 16. Current bestlimit = 15 and setrlimit() failed. Since bestlimit is less than decr_step , then exit the loop. The final bestlimit is larger than oldlimit but is invalid. Note that this only matters if the system fd limit is below 16, so unlikely to have any actual effect.
-
- 23 Aug, 2021 1 commit
-
-
Wen Hui authored
This aims to solve the issue in CONFIG SET maxmemory can only set maxmemory to up to 9223372036854775807 (2^63) while the maxmemory should be ULLONG. Added a memtoull function to convert a string representing an amount of memory into the number of bytes (similar to memtoll but for ull). Also added ull2string to convert a ULLong to string (Similar to ll2string).
-
- 22 Aug, 2021 3 commits
-
-
Viktor Söderqvist authored
Also make sure function can't return NULL by another assert.
-
Binbin authored
In old way, we always increase server.dirty in BITSET and BITFIELD SET. Even the command doesn't really change anything. This commit make sure BITSET and BITFIELD SET only increase dirty when the value changed. Because of that, if the value not changed, some others implications: - Avoid adding useless AOF - Reduce replication traffic - Will not trigger keyspace notifications (setbit) - Will not invalidate WATCH - Will not sent the invalidation message to the tracking client
-
Viktor Söderqvist authored
-
- 20 Aug, 2021 1 commit
-
-
sundb authored
-
- 18 Aug, 2021 1 commit
-
-
yoav-steinberg authored
Following compilation warnings on s390x.
-
- 15 Aug, 2021 1 commit
-
-
Yossi Gottlieb authored
On systems that have unsigned char by default (s390x, arm), redis-server could crash as soon as it populates the command table.
-
- 14 Aug, 2021 1 commit
-
-
Wang Yuan authored
If we want to check `defined(SYNC_FILE_RANGE_WAIT_BEFORE)`, we should include fcntl.h. otherwise, SYNC_FILE_RANGE_WAIT_BEFORE is not defined, and there is alway not `sync_file_range` system call. Introduced by #8532
-
- 12 Aug, 2021 3 commits
-
-
Madelyn Olson authored
-
Madelyn Olson authored
-
Yossi Gottlieb authored
The order of setting things up follows some reasoning: Setup signal handlers first because a signal could fire at any time. Adjust OOM score before everything else to assist the OOM killer if memory resources are low. The trigger for this is a valgrind test failure which resulted with the child catching a SIGUSR1 before initializing the handler.
-
- 10 Aug, 2021 4 commits
-
-
sundb authored
-
Huang Zhw authored
Abort cli blocking modes with SIGINT without exiting the cli. Co-authored-by:
charsyam <charsyam@gmail.com>
-
DarrenJiang13 authored
We only use MADV_DONTNEED on Linux, that's were it was tested.
-
Meir Shpilraien (Spielrein) authored
Following the comments on #8659, this PR fix some formatting and naming issues.
-