- 03 May, 2021 2 commits
-
-
Istemi Ekin Akkus authored
(cherry picked from commit af035c1e)
- 19 Apr, 2021 1 commit
-
-
Hanna Fadida authored
Adding a new type mask for key space notification, REDISMODULE_NOTIFY_MODULE, to enable unique notifications from commands on REDISMODULE_KEYTYPE_MODULE type keys (which is currently unsupported). Modules can subscribe to a module key keyspace notification by RM_SubscribeToKeyspaceEvents, and clients by notify-keyspace-events of redis.conf or via the CONFIG SET, with the characters 'd' or 'A' (REDISMODULE_NOTIFY_MODULE type mask is part of the '**A**ll' notation for key space notifications). Refactor: move some pubsub test infra from pubsub.tcl to util.tcl to be re-used by other tests.
-
- 13 Apr, 2021 3 commits
-
-
Viktor Söderqvist authored
* Modules API docs: Link API function names to their definitions Occurrences of API functions are linked to their definition. A function index with links to all functions is added on the bottom of the page. Comment blocks in module.c starting with a markdown h2 heading are used as sections. A table of contents is generated from these headings. The functions names are changed from h2 to h3, since they are now rendered as sub-headings within each section. Existing sections in module.c are used with some minor changes. Some documentation text is added or sligtly modified. The markdown renderer will add IDs which may clash with our generated IDs. By prefixing section IDs with "section-" we make them different. Replace double dashes with a unicode long ndash
-
Viktor Söderqvist authored
In a code example, using RedisModule_FreeString instead of RedisModule_Free makes it behave correctly regardless of whether automatic memory is used or not.
-
- 06 Apr, 2021 1 commit
-
-
Bonsai authored
With this fix, module data type registration will fail if the load or save callbacks are not defined, or the optional aux load and save callbacks are not either both defined or both missing.
-
- 01 Apr, 2021 1 commit
-
-
Dvir Volk authored
Added macros for RM_log logging levels, to avoid typos and the need to memorize the level strings by heart
-
- 25 Mar, 2021 1 commit
-
-
Oran Agra authored
* SLOWLOG didn't record anything for blocked commands because the client was reset and argv was already empty. there was a fix for this issue specifically for modules, now it works for all blocked clients. * The original command argv (before being re-written) was also reset before adding the slowlog on behalf of the blocked command. * Latency monitor is now updated regardless of the slowlog flags of the command or its execution (their purpose is to hide sensitive info from the slowlog, not hide the fact the latency happened). * Latency monitor now uses real_cmd rather than c->cmd (which may be different if the command got re-written, e.g. GEOADD) Changes: * Unify shared code between slowlog insertion in call() and updateStatsOnUnblock(), hopefully prevent future bugs from happening due to the later being overlooked. * Reset CLIENT_PREVENT_LOGGING in resetClient rather than after command processing. * Add a test for SLOWLOG and BLPOP Notes: - real_cmd == c->lastcmd, except inside MULTI and Lua. - blocked commands never happen in these cases (MULTI / Lua) - real_cmd == c->cmd, except for when the command is rewritten (e.g. GEOADD) - blocked commands (currently) are never rewritten - other than the command's CLIENT_PREVENT_LOGGING, and the execution flag CLIENT_PREVENT_LOGGING, other cases that we want to avoid slowlog are on AOF loading (specifically CMD_CALL_SLOWLOG will be off when executed from execCommand that runs from an AOF)
-
- 24 Mar, 2021 2 commits
-
-
Qu Chen authored
Properly initialize variable to make valgrind happy in checkChildrenDone(). Removed usage for the obsolete wait3() and wait4() in favor of waitpid(), and properly check for the exit status code. (#8666)
-
chenyangyang authored
Add a check to ensure that the expire parameters in RM_SetExpire and RM_SetAbsExpire must be positive.
-
- 16 Mar, 2021 1 commit
-
-
Oran Agra authored
REDISMODULE_CTX_FLAGS_EVICT and REDISMODULE_CTX_FLAGS_MAXMEMORY shouldn't be set when the module is run inside a replica that doesn't do eviction. one may argue that the database is under eviction (the master does eviction and sends DELs to the replica). but on the other hand, we don't really know the master's configuration. all that matters is if the current instance does evictions or not.
-
- 15 Mar, 2021 1 commit
-
-
guybe7 authored
1. moduleReplicateMultiIfNeeded should use server.in_eval like moduleHandlePropagationAfterCommandCallback 2. server.in_eval could have been set to 1 and not reset back to 0 (a lot of missed early-exits after in_eval is already 1) Note: The new assertions in processCommand cover (2) and I added two module tests to cover (1) Implications: If an EVAL that failed (and thus left server.in_eval=1) runs before a module command that replicates, the replication stream will contain MULTI (because moduleReplicateMultiIfNeeded used to check server.lua_caller which is NULL at this point) but not EXEC (because server.in_eval==1) This only affects modules as module.c the only user of server.in_eval. Affects versions 6.2.0, 6.2.1
-
- 14 Mar, 2021 1 commit
-
-
Huang Zhw authored
-
- 10 Mar, 2021 2 commits
-
-
guybe7 authored
Bug 1: When a module ctx is freed moduleHandlePropagationAfterCommandCallback is called and handles propagation. We want to prevent it from propagating commands that were not replicated by the same context. Example: 1. module1.foo does: RM_Replicate(cmd1); RM_Call(cmd2); RM_Replicate(cmd3) 2. RM_Replicate(cmd1) propagates MULTI and adds cmd1 to also_propagagte 3. RM_Call(cmd2) create a new ctx, calls call() and destroys the ctx. 4. moduleHandlePropagationAfterCommandCallback is called, calling alsoPropagates EXEC (Note: EXEC is still not written to socket), setting server.in_trnsaction = 0 5. RM_Replicate(cmd3) is called, propagagting yet another MULTI (now we have nested MULTI calls, which is no good) and then cmd3 We must prevent RM_Call(cmd2) from resetting server.in_transaction. REDISMODULE_CTX_MULTI_EMITTED was revived for that purpose. Bug 2: Fix issues with nested RM_Call where some have '!' and some don't. Example: 1. module1.foo does RM_Call of module2.bar without replication (i.e. no '!') 2. module2.bar internally calls RM_Call of INCR with '!' 3. at the end of module1.foo we call RM_ReplicateVerbatim We want the replica/AOF to see only module1.foo and not the INCR from module2.bar Introduced a global replication_allowed flag inside RM_Call to determine whether we need to replicate or not (even if '!' was specified) Other changes: Split beforePropagateMultiOrExec to beforePropagateMulti afterPropagateExec just for better readability
-
guybe7 authored
Have a clear separation between in and out flags Other changes: delete dead code in RM_ZsetIncrby: if zsetAdd returned error (happens only if the result of the operation is NAN or if score is NAN) we return immediately so there is no way that zsetAdd succeeded and returned NAN in the out-flags
-
- 08 Mar, 2021 1 commit
-
-
guybe7 authored
Scenario: 1. A module client is blocked on keys with a timeout 2. Shortly before the timeout expires, the key is being populated and signaled as ready 3. Redis calls moduleTryServeClientBlockedOnKey (which replies to client) and then moduleUnblockClient 4. moduleUnblockClient doesn't really unblock the client, it writes to server.module_blocked_pipe and only marks the BC as unblocked. 5. beforeSleep kics in, by this time the client still exists and techincally timed-out. beforeSleep replies to the timeout client (double reply) and only then moduleHandleBlockedClients is called, reading from module_blocked_pipe and calling unblockClient The solution is similar to what was done in moduleTryServeClientBlockedOnKey: we should avoid re-processing an already-unblocked client
-
- 02 Mar, 2021 1 commit
-
-
Yossi Gottlieb authored
Fixes #8574
-
- 28 Feb, 2021 2 commits
-
-
Bonsai authored
Adding RM_GetClientUserNameById to get the ACL user name of a client connection.
-
Viktor Söderqvist authored
A single client pointer is added in the server struct. This is initialized by the first RM_Call() and reused for every subsequent RM_Call() except if it's already in use, which means that it's not used for (recursive) module calls to modules. For these, a new "fake" client is created each time. Other changes: * Avoid allocating a dict iterator in pubsubUnsubscribeAllChannels when not needed
-
- 16 Feb, 2021 1 commit
-
-
uriyage authored
* Adding current_save_keys_total and current_save_keys_processed info fields. Present in replication, BGSAVE and AOFRW. * Changing RM_SendChildCOWInfo() to RM_SendChildHeartbeat(double progress) * Adding new info field current_fork_perc. Present in Replication, BGSAVE, AOFRW, and module forks.
-
- 15 Feb, 2021 2 commits
-
-
Yossi Gottlieb authored
Fixes #8489
-
Viktor Söderqvist authored
The added flag affects the return value of RM_HashSet() to include the number of inserted fields, in addition to updated and deleted fields. errno is set on errors, tests are added and documentation updated.
-
- 10 Feb, 2021 1 commit
-
-
filipe oliveira authored
- removes time sensitive checks from block on background tests during leak checks. - fix uninitialized variable on RedisModuleBlockedClient() when calling RM_BlockedClientMeasureTimeEnd() without RM_BlockedClientMeasureTimeStart()
-
- 05 Feb, 2021 1 commit
-
-
Viktor Söderqvist authored
Without this fix, RM_ZsetRem can leave empty sorted sets which are not allowed to exist. Removing from a sorted set while iterating seems to work (while inserting causes failed assetions). RM_ZsetRangeEndReached is modified to return 1 if the key doesn't exist, to terminate iteration when the last element has been removed.
-
- 29 Jan, 2021 1 commit
-
-
filipe oliveira authored
Enabled background and reply time tracking on blocked on keys/blocked on background work clients (#7491) This commit enables tracking time of the background tasks and on replies, opening the door for properly tracking commands that rely on blocking / background work via the slowlog, latency history, and commandstats. Some notes: - The time spent blocked waiting for key changes, or blocked on synchronous replication is not accounted for. - **This commit does not affect latency tracking of commands that are non-blocking or do not have background work.** ( meaning that it all stays the same with exception to `BZPOPMIN`,`BZPOPMAX`,`BRPOP`,`BLPOP`, etc... and module's commands that rely on background threads ). - Specifically for latency history command we've added a new event class named `command-unblocking` that will enable latency monitoring on commands that spawn background threads to do the work. - For blocking commands we're now considering the total time of a command as the time spent on call() + the time spent on replying when unblocked. - For Modules commands that rely on background threads we're now considering the total time of a command as the time spent on call (main thread) + the time spent on the background thread ( if marked within `RedisModule_MeasureTimeStart()` and `RedisModule_MeasureTimeEnd()` ) + the time spent on replying (main thread) To test for this feature we've added a `unit/moduleapi/blockonbackground` test that relies on a module that blocks the client and sleeps on the background for a given time. - check blocked command that uses RedisModule_MeasureTimeStart() is tracking background time - check blocked command that uses RedisModule_MeasureTimeStart() is tracking background time even in timeout - check blocked command with multiple calls RedisModule_MeasureTimeStart() is tracking the total background time - check blocked command without calling RedisModule_MeasureTimeStart() is not reporting background time
-
- 28 Jan, 2021 2 commits
-
-
guybe7 authored
Useful to avoid doing background jobs that can cause excessive COW
-
Viktor Söderqvist authored
APIs added for these stream operations: add, delete, iterate and trim (by ID or maxlength). The functions are prefixed by RM_Stream. * RM_StreamAdd * RM_StreamDelete * RM_StreamIteratorStart * RM_StreamIteratorStop * RM_StreamIteratorNextID * RM_StreamIteratorNextField * RM_StreamIteratorDelete * RM_StreamTrimByLength * RM_StreamTrimByID The type RedisModuleStreamID is added and functions for converting from and to RedisModuleString. * RM_CreateStringFromStreamID * RM_StringToStreamID Whenever the stream functions return REDISMODULE_ERR, errno is set to provide additional error information. Refactoring: The zset iterator fields in the RedisModuleKey struct are wrapped in a union, to allow the same space to be used for type- specific info for streams and allow future use for other key types.
-
- 20 Jan, 2021 1 commit
-
-
Viktor Söderqvist authored
Fix broken formatting in `RM_Call` and `RM_CreateDataType`, `RM_SubscribeToServerEvent` (nested lists, etc. in list items). Unhide docs of `RM_LoadDataTypeFromString` and `RM_SaveDataTypeToString` by removing blank line between docs and function. Clarification added to `RM__Assert`: Recommentation to use the `RedisModule_Assert` macro instead. All names containing underscores (variable and macro names) are wrapped in backticks (if not already wrapped in backticks). This prevents underscore from being interpreted as italics in some cases. Names including a wildcard star, e.g. RM_Defrag*(), is wrapped in backticks (and RM replaced by RedisModule in this case). This prevents the * from being interpreted as an italics marker. A list item with a sublist, a paragraph and another sublist is a combination which seems impossible to achieve with RedCarped markdown, so the one occurrence of this is rewritten. Various trivial changes (typos, backticks, etc.). Ruby script: * Replace `RM_Xyz` with `RedisModule_Xyz` in docs. (RM is correct when refering to the C code but RedisModule is correct in the API docs.) * Automatic backquotes around C functions like `malloc()`. * Turn URLs into links. The link text is the URL itself. * Don't add backticks inside bold (**...**)
-
- 19 Jan, 2021 1 commit
-
-
Andy Pan authored
Sentinel uses execve to run scripts, so it needs to use FD_CLOEXEC on all file descriptors, so that they're not accessible by the script it runs. This commit includes a change to the sentinel tests, which verifies no FDs are left opened when the script is executed.
-
- 15 Jan, 2021 2 commits
-
-
Viktor Söderqvist authored
The prefix is changed from `RM_` to `module` on the following internal functions, to prevent them from appearing in the API docs: RM_LogRaw -> moduleLogRaw RM_FreeCallReplyRec -> moduleFreeCallReplyRec RM_ZsetAddFlagsToCoreFlags -> moduleZsetAddFlagsToCoreFlags RM_ZsetAddFlagsFromCoreFlags -> moduleZsetAddFlagsFromCoreFlags
-
Viktor Söderqvist authored
Fixes markdown formatting errors and some functions not showing up in the generated documentation at all. Ruby script (gendoc.rb) fixes: * Modified automatic instertion of backquotes: * Don't add backquotes around names which are already preceded by a backquote. Fixes for example \`RedisModule_Reply\*\` which turning into \`\`RedisModule_Reply\`\*\` messes up the formatting. * Add backquotes around types such as RedisModuleString (in addition to function names `RedisModule_[A-z()]*` and macro names `REDISMODULE_[A-z]*`). * Require 4 spaces indentation for disabling automatic backquotes, i.e. code blocks. Fixes continuations of list items (indented 2 spaces). * More permissive extraction of doc comments: * Allow doc comments starting with `/**`. * Make space before `*` on each line optional. * Make space after `/*` and `/**` optional (needed when appearing on its own line). Markdown fixes in module.c: * Fix code blocks not indented enough (4 spaces needed). * Add black line before code blocks and lists where missing (needed). * Enclose special markdown characters `_*^<>` in backticks to prevent them from messing up formatting. * Lists with `1)` changed to `1.` for proper markdown lists. * Remove excessive indentation which causes text to be unintentionally rendered as code blocks. * Other minor formatting fixes. Other fixes in module.c: * Remove blank lines between doc comment and function definition. A blank line here makes the Ruby script exclude the function in docs.
-
- 08 Jan, 2021 1 commit
-
-
Madelyn Olson authored
Implementation of client pause WRITE and client unpause
-
- 07 Jan, 2021 2 commits
-
-
YaacovHazan authored
Add INFO field, rdb_active_cow_size, to report COW of a live fork child while it's active. - once in 1024 keys check the time, and if there's more than one second since the last report send a report to the parent via the pipe. - refactor the child_info_data struct, it's an implementation detail that shouldn't be in the server struct, and not used to communicate data between caller and callee - remove the magic value from that struct (not sure what it was good for), and instead add handling of short reads. - add another value to the structure, cow_type, to indicate if the report is for the new rdb_active_cow_size field, or it's the last report of a successful operation - add new Module API to report the active COW - add more asserts variants to test.tcl
-
YaacovHazan authored
This is a refactory commit, isn't suppose to have any actual impact. it does the following: - keep just one server struct fork child pid variable instead of 3 - have one server struct variable indicating the purpose of the current fork child. - redisFork is now responsible of updating the server struct with the pid, which means it can be the one that calls updateDictResizePolicy - move child info pipe handling into redisFork instead of having them repeated outside - there are two classes of fork purposes, mutually exclusive group (AOF, RDB, Module), and one that can create several forks to coexist in parallel (LDB, but maybe Modules some day too, Module API allows for that). - minor fix to killRDBChild: unlike killAppendOnlyChild and TerminateModuleForkChild, the killRDBChild doesn't clear the pid variable or call wait4, so checkChildrenDone does the cleanup for it. This commit removes the explicit calls to rdbRemoveTempFile, closeChildInfoPipe, updateDictResizePolicy, which didn't do any harm, but where unnecessary.
-
- 04 Jan, 2021 1 commit
-
-
Itamar Haber authored
* man-like consistent long formatting * Uppercases commands, subcommands and options * Adds 'HELP' to HELP for all * Lexicographical order * Uses value notation and other .md likeness * Moves const char *help to top * Keeps it under 80 chars * Misc help typos, consistent conjuctioning (i.e return and not returns) * Uses addReplySubcommandSyntaxError(c) all over Signed-off-by:
Itamar Haber <itamar@redislabs.com>
-
- 03 Jan, 2021 1 commit
-
-
Oran Agra authored
-
- 31 Dec, 2020 1 commit
-
-
filipe oliveira authored
This Commit pushes forward the observability on overall error statistics and command statistics within redis-server: It extends INFO COMMANDSTATS to have - failed_calls in - so we can keep track of errors that happen from the command itself, broken by command. - rejected_calls - so we can keep track of errors that were triggered outside the commmand processing per se Adds a new section to INFO, named ERRORSTATS that enables keeping track of the different errors that occur within redis ( within processCommand and call ) based on the reply Error Prefix ( The first word after the "-", up to the first space ). This commit also fixes RM_ReplyWithError so that it can be correctly identified as an error reply.
-
- 22 Dec, 2020 1 commit
-
-
Oran Agra authored
Remove read-only flag from non-keyspace cmds, different approach for EXEC to propagate MULTI (#8216) In the distant history there was only the read flag for commands, and whatever command that didn't have the read flag was a write one. Then we added the write flag, but some portions of the code still used !read Also some commands that don't work on the keyspace at all, still have the read flag. Changes in this commit: 1. remove the read-only flag from TIME, ECHO, ROLE and LASTSAVE 2. EXEC command used to decides if it should propagate a MULTI by looking at the command flags (!read & !admin). When i was about to change it to look at the write flag instead, i realized that this would cause it not to propagate a MULTI for PUBLISH, EVAL, and SCRIPT, all 3 are not marked as either a read command or a write one (as they should), but all 3 are calling forceCommandPropagation. So instead of introducing a new flag to denote a command that "writes" but not into the keyspace, and still needs propagation, i decided to rely on the forceCommandPropagation, and just fix the code to propagate MULTI when needed rather than depending on the command flags at all. The implication of my change then is that now it won't decide to propagate MULTI when it sees one of these: SELECT, PING, INFO, COMMAND, TIME and other commands which are neither read nor write. 3. Changing getNodeByQuery and clusterRedirectBlockedClientIfNeeded in cluster.c to look at !write rather than read flag. This should have no implications, since these code paths are only reachable for commands which access keys, and these are always marked as either read or write. This commit improve MULTI propagation tests, for modules and a bunch of other special cases, all of which used to pass already before that commit. the only one that test change that uncovered a change of behavior is the one that DELs a non-existing key, it used to propagate an empty multi-exec block, and no longer does.
-
- 13 Dec, 2020 1 commit
-
-
Yossi Gottlieb authored
* Allow runtest-moduleapi use a different 'make', for systems where GNU Make is 'gmake'. * Fix issue with builds on Solaris re-building everything from scratch due to CFLAGS/LDFLAGS not stored. * Fix compile failure on Solaris due to atomicvar and a bunch of warnings. * Fix garbled log timestamps on Solaris.
-