- 15 Dec, 2020 1 commit
-
-
sundb authored
Fix wrong server dirty increment in * spopWithCountCommand * hsetCommand * ltrimCommand * pfaddCommand Some didn't increment the amount of fields (just one per command). Others had excessive increments.
-
- 14 Dec, 2020 2 commits
-
-
Itamar Haber authored
And formats style a tiniee-winiee bit
-
Oran Agra authored
-
- 13 Dec, 2020 10 commits
-
-
filipe oliveira authored
Exposes the main thread CPU info via info modules ( linux specific only ) (used_cpu_sys_main_thread and used_cpu_user_main_thread). This is important for: - distinguish between main thread and io-threads cpu time total cpu time consumed ( check what is the first bottleneck on the used config ) - distinguish between main thread and modules threads total cpu time consumed Apart from it, this commit also exposes the server_time_usec within the Server section so that we can properly differentiate consecutive collection and calculate for example the CPU% and or / cpu time vs wall time, etc...
-
Yossi Gottlieb authored
* Allow runtest-moduleapi use a different 'make', for systems where GNU Make is 'gmake'. * Fix issue with builds on Solaris re-building everything from scratch due to CFLAGS/LDFLAGS not stored. * Fix compile failure on Solaris due to atomicvar and a bunch of warnings. * Fix garbled log timestamps on Solaris.
-
Wen Hui authored
-
Oran Agra authored
When a replica uses the diskless-load swapdb approach, it backs up the old database, then attempts to load a new one, and in case of failure, it restores the backup. this means that modules with global out of keyspace data, must have an option to subscribe to events and backup/restore/discard their global data too.
-
Wen Hui authored
-
Madelyn Olson authored
-
Wen Hui authored
without this option, redis-cli returns 0 even if command fails. kept this for backwards compatibility.
-
Wang Yuan authored
-
Yossi Gottlieb authored
Add a new set of defrag functions that take a defrag context and allow defragmenting memory blocks and RedisModuleStrings. Modules can register a defrag callback which will be invoked when the defrag process handles globals. Modules with custom data types can also register a datatype-specific defrag callback which is invoked for keys that require defragmentation. The callback and associated functions support both one-step and multi-step options, depending on the complexity of the key as exposed by the free_effort callback.
-
gourav authored
The pid of the benchmark process is used to randomize the random number generator's seed. This ensures that when multiple benchmark processes are started at the same time to generate load on a server, they use different seeds. This will ensure randomness in the keys generated by different benchmark processes.
-
- 12 Dec, 2020 1 commit
-
-
杨博东 authored
Add commands to query geospatial data with bounding box. Two new commands that replace the existing 4 GEORADIUS* commands. GEOSEARCH key [FROMMEMBER member] [FROMLOC long lat] [BYRADIUS radius unit] [BYBOX width height unit] [WITHCORD] [WITHDIST] [WITHASH] [COUNT count] [ASC|DESC] GEOSEARCHSTORE dest_key src_key [FROMMEMBER member] [FROMLOC long lat] [BYRADIUS radius unit] [BYBOX width height unit] [WITHCORD] [WITHDIST] [WITHASH] [COUNT count] [ASC|DESC] [STOREDIST] - Add two types of CIRCULAR_TYPE and RECTANGLE_TYPE to achieve different searches - Judge whether the point is within the rectangle, refer to: geohashGetDistanceIfInRectangle
-
- 11 Dec, 2020 1 commit
-
-
Yossi Gottlieb authored
This adds a new `tls-client-cert-file` and `tls-client-key-file` configuration directives which make it possible to use different certificates for the TLS-server and TLS-client functions of Redis. This is an optional directive. If it is not specified the `tls-cert-file` and `tls-key-file` directives are used for TLS client functions as well. Also, `utils/gen-test-certs.sh` now creates additional server-only and client-only certs and will skip intensive operations if target files already exist.
-
- 09 Dec, 2020 3 commits
-
-
Yossi Gottlieb authored
This adds a copy callback for module data types, in order to make modules compatible with the new COPY command. The callback is optional and COPY will fail for keys with data types that do not implement it.
-
Yossi Gottlieb authored
Fixes #8165.
-
Oran Agra authored
-
- 08 Dec, 2020 4 commits
-
-
Oran Agra authored
Module blocked clients cache the response in a temporary client, the reply list in this client would be affected by the recent fix in #7202, but when the reply is later copied into the real client, it would have bypassed all the checks for output buffer limit, which would have resulted in both: responding with a partial response to the client, and also not disconnecting it at all.
-
Itamar Haber authored
-
Yossi Gottlieb authored
Makes spt_init more careful with assumptions about what memory regions may be overwritten. It will now only consider a contiguous block of argv and envp elements and mind any gaps.
-
Wen Hui authored
This is a cleanup commit that doesn't fix any bug, however it does change the error response text in case the user provided a negative number.
-
- 07 Dec, 2020 3 commits
-
-
guybe7 authored
when the same consumer re-claim an entry that it already has, there's no need to remove-and-insert if it's the same rax. we do need to update the idle time though. this commit only improves efficiency (doesn't change behavior).
-
filipe oliveira authored
-
Yossi Gottlieb authored
* Add CLIENT INFO subcommand. The output is identical to CLIENT LIST but provides a single line for the current client only. * Add CLIENT LIST ID [id...]. Co-authored-by:
Itamar Haber <itamar@redislabs.com>
-
- 06 Dec, 2020 9 commits
-
-
David CARLIER authored
-
Oran Agra authored
First, if the ziplist header is surely inside the ziplist, do fast path decoding rather than the careful one. In that case, streamline the encoding if-else chain to be executed only once, and the encoding validity tested at the end. encourage inlining likely / unlikely hints for speculative execution Assertion used _exit(1) to tell the compiler that the code after them is not reachable and get rid of warnings. But in some cases assertions are placed inside tight loops, and any piece of code in them can slow down execution (code cache and other reasons), instead using either abort() or better yet, unreachable builtin.
-
Oran Agra authored
When RDB input attempts to make a huge memory allocation that fails, RESTORE should fail gracefully rather than die with panic
-
Oran Agra authored
If RESTORE passes successfully with full sanitization, we can't affort to crash later on assertion due to duplicate records in a hash when converting it form ziplist to dict. This means that when doing full sanitization, we must make sure there are no duplicate records in any of the collections.
-
Oran Agra authored
The test creates keys with various encodings, DUMP them, corrupt the payload and RESTORES it. It utilizes the recently added use-exit-on-panic config to distinguish between asserts and segfaults. If the restore succeeds, it runs random commands on the key to attempt to trigger a crash. It runs in two modes, one with deep sanitation enabled and one without. In the first one we don't expect any assertions or segfaults, in the second one we expect assertions, but no segfaults. We also check for leaks and invalid reads using valgrind, and if we find them we print the commands that lead to that issue. Changes in the code (other than the test): - Replace a few NPD (null pointer deference) flows and division by zero with an assertion, so that it doesn't fail the test. (since we set the server to use `exit` rather than `abort` on assertion). - Fix quite a lot of flows in rdb.c that could have lead to memory leaks in RESTORE command (since it now responds with an error rather than panic) - Add a DEBUG flag for SET-SKIP-CHECKSUM-VALIDATION so that the test don't need to bother with faking a valid checksum - Remove a pile of code in serverLogObjectDebugInfo which is actually unsafe to run in the crash report (see comments in the code) - fix a missing boundary check in lzf_decompress test suite infra improvements: - be able to run valgrind checks before the process terminates - rotate log files when restarting servers
-
Oran Agra authored
When loading an encoded payload we will at least do a shallow validation to check that the size that's encoded in the payload matches the size of the allocation. This let's us later use this encoded size to make sure the various offsets inside encoded payload don't reach outside the allocation, if they do, we'll assert/panic, but at least we won't segfault or smear memory. We can also do 'deep' validation which runs on all the records of the encoded payload and validates that they don't contain invalid offsets. This lets us detect corruptions early and reject a RESTORE command rather than accepting it and asserting (crashing) later when accessing that payload via some command. configuration: - adding ACL flag skip-sanitize-payload - adding config sanitize-dump-payload [yes/no/clients] For now, we don't have a good way to ensure MIGRATE in cluster resharding isn't being slowed down by these sanitation, so i'm setting the default value to `no`, but later on it should be set to `clients` by default. changes: - changing rdbReportError not to `exit` in RESTORE command - adding a new stat to be able to later check if cluster MIGRATE isn't being slowed down by sanitation.
-
Oran Agra authored
When client tracking is enabled signalModifiedKey can increase memory usage, this can cause the loop in performEvictions to keep running since it was measuring the memory usage impact of signalModifiedKey. The section that measures the memory impact of the eviction should be just on dbDelete, excluding keyspace notification, client tracking, and propagation to AOF and replicas. This resolves part of the problem described in #8069 p.s. fix took 1 minute, test took about 3 hours to write.
-
guybe7 authored
One way this was happening is when a module issued an RM_Call which would inject MULTI. If the module command that does that was itself issued by something else that already did added MULTI (e.g. another module, or a Lua script), it would have caused nested MULTI. In fact the MULTI state in the client or the MULTI_EMITTED flag in the context isn't the right indication that we need to propagate MULTI or not, because on a nested calls (possibly a module action called by a keyspace event of another module action), these flags aren't retained / reflected. instead there's now a global propagate_in_transaction flag for that. in addition to that, we now have a global in_eval and in_exec flags, to serve the flags of RM_GetContextFlags, since their dependence on the current client is wrong for the same reasons mentioned above.
-
Wang Yuan authored
As we know, redis may reject user's requests or evict some keys if used memory is over maxmemory. Dictionaries expanding may make things worse, some big dictionaries, such as main db and expires dict, may eat huge memory at once for allocating a new big hash table and be far more than maxmemory after expanding. There are related issues: #4213 #4583 More details, when expand dict in redis, we will allocate a new big ht[1] that generally is double of ht[0], The size of ht[1] will be very big if ht[0] already is big. For db dict, if we have more than 64 million keys, we need to cost 1GB for ht[1] when dict expands. If the sum of used memory and new hash table of dict needed exceeds maxmemory, we shouldn't allow the dict to expand. Because, if we enable keys eviction, we still couldn't add much more keys after eviction and rehashing, what's worse, redis will keep less keys when redis only remains a little memory for storing new hash table instead of users' data. Moreover users can't write data in redis if disable keys eviction. What this commit changed ? Add a new member function expandAllowed for dict type, it provide a way for caller to allow expand or not. We expose two parameters for this function: more memory needed for expanding and dict current load factor, users can implement a function to make a decision by them. For main db dict and expires dict type, these dictionaries may be very big and cost huge memory for expanding, so we implement a judgement function: we can stop dict to expand provisionally if used memory will be over maxmemory after dict expands, but to guarantee the performance of redis, we still allow dict to expand if dict load factor exceeds the safe load factor. Add test cases to verify we don't allow main db to expand when left memory is not enough, so that avoid keys eviction. Other changes: For new hash table size when expand. Before this commit, the size is that double used of dict and later _dictNextPower. Actually we aim to control a dict load factor between 0.5 and 1.0. Now we replace *2 with +1, since the first check is that used >= size, the outcome of before will usually be the same as _dictNextPower(used+1). The only case where it'll differ is when dict_can_resize is false during fork, so that later the _dictNextPower(used*2) will cause the dict to jump to *4 (i.e. _dictNextPower(1025*2) will return 4096). Fix rehash test cases due to changing algorithm of new hash table size when expand.
-
- 03 Dec, 2020 3 commits
-
-
Itamar Haber authored
Adds the ability to use exclusive (open) start and end query intervals in XRANGE and XREVRANGE queries. Fixes #6562
-
Felipe Machado authored
In the iterator for these functions, we'll traverse the sorted sets in a reversed way so that largest elements come first. We prefer this order because it's optimized for insertion in a skiplist, which is the destination of the elements being iterated in there functions.
- 02 Dec, 2020 2 commits
-
-
Wang Yuan authored
Backup keys to slots map and restore when fail to sync if diskless-load type is swapdb in cluster mode (#8108) When replica diskless-load type is swapdb in cluster mode, we didn't backup keys to slots map, so we will lose keys to slots map if fail to sync. Now we backup keys to slots map at first, and restore it properly when fail. This commit includes a refactory/cleanup of the backups mechanism (moving it to db.c and re-structuring it a bit). Co-authored-by:
Oran Agra <oran@redislabs.com>
-
luhuachao authored
As described in redis-benchamrk help message 'The test names are the same as the ones produced as output.', In redis-benchmark output, we can only see PING_BULK, but the cmd `redis-benchmark -t ping_bulk` is not supported. We have to run it with ping_mbulk which is not user friendly.
-
- 01 Dec, 2020 1 commit
-
-
Madelyn Olson authored
* Fixed SET GET executing on wrong type Co-authored-by:
Madelyn Olson <madelyneolson@gmail.com>
-