- 22 May, 2022 4 commits
-
-
yoav-steinberg authored
The code is based on upstream https://github.com/HdrHistogram/HdrHistogram_c master branch latest commit (e4448cf6d1cd08fff519812d3b1e58bd5a94ac42). The reason to pull this in now is that their final version of our optimization is even faster. See: https://github.com/HdrHistogram/HdrHistogram_c/pull/107.
-
Binbin authored
Remove some dead code in object.c, ziplist is no longer used in 7.0 Some backgrounds: zipmap - hash: replaced by ziplist in #285 ziplist - hash: replaced by listpack in #8887 ziplist - zset: replaced by listpack in #9366 ziplist - list: replaced by quicklist (listpack) in #2143 / #9740 Moved the location of ziplist.h in the server.c
-
Yuuoniy authored
si is initialized by streamIteratorStart(), we should call streamIteratorStop() on it when done. regression introduced in #9127 (redis 7.0)
-
Ofir Luzon authored
Finish current loop and display the scanned keys summery on SIGINT (Ctrl-C) signal. It will also prepend the current scanned percentage to the scanned keys summery 1st line. In this commit I've renamed and relocated `intrinsicLatencyModeStop` function as I'm using the exact same logic.
-
- 18 May, 2022 1 commit
-
-
Binbin authored
minor cleanup in redismodule.h and module.c
-
- 16 May, 2022 2 commits
-
-
Ozan Tezcan authored
was present in the C file and missing in the header.
-
Qu Chen authored
We observed from our replication testing that when the master becomes unresponsive, or the replication connection is broken during PSYNC so the replica doesn't get a response from the master, it was not able to recognize that condition as a failure and instead moved into the full-sync code path. This fix makes the replica fail and retry the PSYNC with the master in such scenarios.
-
- 15 May, 2022 2 commits
- 13 May, 2022 2 commits
-
-
Stephen Sullivan authored
Alias was mistakenly forgotten when the sub commands introduced as json files.
-
Wen Hui authored
Updated the comments for: info command lmpopCommand and blmpopCommand sinterGenericCommand Fix the missing "key" words in the srandmemberCommand function For LPOS command, when rank is 0, prompt user that rank could be positive number or negative number, and add a test for it
-
- 12 May, 2022 2 commits
-
-
Binbin authored
The purpose of the test is to kill the child while it is running. From the last two lines we can see the child exits before being killed. ``` - Module fork started pid: 56998 * <fork> fork child started - Killing running module fork child: 56998 * <fork> fork child exiting signal-handler (1652267501) Received SIGUSR1 in child, exiting now. ``` In this commit, we pass an argument to `fork.create` indicating how long it should sleep. For the fork kill test, we use a longer time to avoid the child exiting before being killed. Other changes: use wait_for_condition instead of hardcoded `after 250`. Unify the test for failing fork with the one for killing it (save time)
-
yoav-steinberg authored
Fix possible regression around TLS config changes. Add VOLATILE_CONFIG flag for volatile configurations. (#10713) This fixes a possible regression in Redis 7.0.0, in which doing CONFIG SET on a TLS config would not reload the configuration in case the new config is the same file as before. A volatile configuration is a configuration value which is a reference to the configuration data and not the configuration data itself. In such a case Redis doesn't know if the config data changed under the hood and can't assume a change happens only when the config value changes. Therefore it needs to be applied even when setting a config value to the same value as it was before.
-
- 11 May, 2022 4 commits
-
-
Yossi Gottlieb authored
-
Yossi Gottlieb authored
Before this commit, all source files including those that are not going to be compiled were used. Some of these files are platform specific and won't even pre-process on another platform. With GCC/Clang, that's not an issue and they'll simply ignore them, but ICC aborts in this case. This commit only attempts to generate Makefile.dep from the actual set of C source files that will be compiled.
-
Binbin authored
redis-server command line arguments support take one bulk string with spaces for MULTI_ARG configs parsing. And allow options value to use the -- prefix (#10660) ## Take one bulk string with spaces for MULTI_ARG configs parsing Currently redis-server looks for arguments that start with `--`, and anything in between them is considered arguments for the config. like: `src/redis-server --shutdown-on-sigint nosave force now --port 6380` MULTI_ARG configs behave differently for CONFIG command, vs the command line argument for redis-server. i.e. CONFIG command takes one bulk string with spaces in it, while the command line takes an argv array with multiple values. In this PR, in config.c, if `argc > 1` we can take them as is, and if the config is a `MULTI_ARG` and `argc == 1`, we will split it by spaces. So both of these will be the same: ``` redis-server --shutdown-on-sigint nosave force now --shutdown-on-sigterm nosave force redis-server --shutdown-on-sigint nosave "force now" --shutdown-on-sigterm nosave force redis-server --shutdown-on-sigint nosave "force now" --shutdown-on-sigterm "nosave force" ``` ## Allow options value to use the `--` prefix Currently it decides to switch to the next config, as soon as it sees `--`, even if there was not a single value provided yet to the last config, this makes it impossible to define a config value that has `--` prefix in it. For instance, if we want to set the logfile to `--my--log--file`, like `redis-server --logfile --my--log--file --loglevel verbose`, current code will handle that incorrectly. In this PR, now we allow a config value that has `--` prefix in it. **But note that** something like `redis-server --some-config --config-value1 --config-value2 --loglevel debug` would not work, because if you want to pass a value to a config starting with `--`, it can only be a single value. like: `redis-server --some-config "--config-value1 --config-value2" --loglevel debug` An example (using `--` prefix config value): ``` redis-server --logfile --my--log--file --loglevel verbose redis-cli config get logfile loglevel 1) "loglevel" 2) "verbose" 3) "logfile" 4) "--my--log--file" ``` ### Potentially breaking change `redis-server --save --loglevel verbose` used to work the same as `redis-server --save "" --loglevel verbose` now, it'll error!
-
Binbin authored
FLUSHDB and FLUSHALL add call forceCommandPropagation / FLUSHALL reset dirty counter to 0 if we enable save (#10691) ## FLUSHALL We used to restore the dirty counter after `rdbSave` zeroed it if we enable save. Otherwise FLUSHALL will not be replicated nor put into the AOF. And then we do increment it again below. Without that extra dirty++, when db was already empty, FLUSHALL will not be replicated nor put into the AOF. We now gonna replace all that dirty counter magic with a call to forceCommandPropagation (REPL and AOF), instead of all the messing around with the dirty counter. Added tests to cover three part (dirty counter, REPL, AOF). One benefit other than cleaner code is that the `rdb_changes_since_last_save` is correct in this case. ## FLUSHDB FLUSHDB was not replicated nor put into the AOF when db was already empty. Unlike DEL on a non-existing key, FLUSHDB always does something, and that's to call the module hook. So basically FLUSHDB is never a NOP, and thus it should always be propagated. Not doing that, could mean that if a module does something in that hook, and wants to avoid issues of that hook being missing on the replica if the db is empty, it'll need to do complicated things. So now FLUSHDB add call forceCommandPropagation, we will always propagate FLUSHDB. Always propagating FLUSHDB seems like a safe approach that shouldn't have any drawbacks (other than looking odd) This was mentioned in #8972 ## Test section: We actually found it while solving a race condition in the BGSAVE test (other.tcl). It was found in extra_ci Daily Arm64 (test-libc-malloc). ``` [exception]: Executing test client: ERR Background save already in progress. ERR Background save already in progress ``` It look like `r flushdb` trigger (schedule) a bgsave right after `waitForBgsave r` and before `r save`. Changing flushdb to flushall, FLUSHALL will do a foreground save and then set the dirty counter to 0.
-
- 10 May, 2022 4 commits
-
-
guybe7 authored
Fix #10552 We no longer piggyback getkeys_proc to hold the RedisModuleCommand struct, when exists Others: Use `doesCommandHaveKeys` in `RM_GetCommandKeysWithFlags` and `getKeysSubcommandImpl`. It causes a very minor behavioral change in commands that don't have actual keys, but have a spec with `CMD_KEY_NOT_KEY`. For example, before this command `COMMAND GETKEYS SPUBLISH` would return `Invalid arguments specified for command` but not it returns `The command has no key arguments`
-
Mariya Markova authored
I suggest to use "[fpclassify](https://en.cppreference.com/w/cpp/numeric/math/fpclassify)" for float comparison with zero, because of expression "value == 0" with value very close to zero can be considered as true with some performance compiler optimizations. Note: this code was introduced by 9d520a7f to accept zset scores that get ERANGE in conversion due to precision loss near 0. But with Intel compilers, ICC and ICX, where optimizations for 0 check are more aggressive, "==0" is true for mentioned functions, however should not be. Behavior is seen starting from O2. This leads to a failure in the ZSCAN test in scan.tcl
-
Binbin authored
It used to returns slots as strings, like: ``` redis> cluster shards 1) 1) "slots" 2) 1) "10923" 2) "16383" ``` CLUSTER SHARDS docs and the top comment of #10293 says that it returns integers. Note other commands like CLUSTER SLOTS, it returns slots as integers. Use addReplyLongLong instead of addReplyBulkLongLong, now it returns slots as integers: ``` redis> cluster shards 1) 1) "slots" 2) 1) (integer) 10923 2) (integer) 16383 ``` This is a small breaking change, introduced in 7.0.0 (7.0 RC3, #10293) Fixes #10680
-
Meir Shpilraien (Spielrein) authored
Set `old_li` to NULL to avoid linking it again on error. Before the fix, loading an already existing library will cause the existing library to be added again. This cause not harm other then wrong statistics. The statistics that are effected by the issue are: * `libraries_count` and `functions_count` returned by `function stats` command * `used_memory_functions` returned on `info memory` command * `functions.caches` returned on `memory stats` command
-
- 09 May, 2022 5 commits
-
-
Ozan Tezcan authored
Changed cursor's type from `int` to `unsigned long` allows handling database or key with more than 2^31 elements
-
Lu JJ authored
fix some typo in "t_zset.c". 1. `zzlisinlexrange` the function name mentioned in the comment is misspelled. 2. fix typo in function name`zarndmemberReplyWithListpack` -> `zrandmemberReplyWithListpack`
-
Oran Agra authored
Unintentional change in #9644 (since RC1) meant that an empty `--save ""` config from command line, wouldn't have clear any setting from the config file Added tests to cover that, and improved test infra to take additional command line args for redis-server
-
Oran Agra authored
If we want to support bits that can be overlapping, we need to make sure that: 1. we don't use the same bit for two return values. 2. values should be sorted so that prefer ones (matching more bits) come first.
-
Ozan Tezcan authored
Fixed RM_Scan() usage example: `RedisModuleCursor` -> `RedisModuleScanCursor`
-
- 08 May, 2022 2 commits
-
-
David CARLIER authored
also fixing already defined constants build warning while at it. Co-authored-by:
Oran Agra <oran@redislabs.com>
-
Shaya Potter authored
this seems to have been an oversight. syncing the flags so that NOTIFY_NEW is available to modules. missing in #10512
-
- 04 May, 2022 1 commit
-
-
dependabot[bot] authored
* Bump github/codeql-action from 1 to 2 Bumps [github/codeql-action](https://github.com/github/codeql-action) from 1 to 2. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](https://github.com/github/codeql-action/compare/v1...v2 ) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by:
dependabot[bot] <support@github.com> * Avoid CodeQL on push error. Co-authored-by:
dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
-
- 03 May, 2022 2 commits
-
-
Viktor Söderqvist authored
* Module API doc script: Mark unreleased API functions * fix broken quotes in generate-module-api-doc.rb Co-authored-by:
Oran Agra <oran@redislabs.com>
-
Meir Shpilraien (Spielrein) authored
In general, our error handler make sure the error object is always a table. In some rare cases (such as OOM error), the error handler will not be called and the error object will be a string. The PR expose the error even if its a string and not a table. Currently there is no way to test it but if it'll ever happen, it is better to propagate this string upwards than just generate a generic error without any specific info.
-
- 02 May, 2022 3 commits
-
-
Lu JJ authored
When user uses the same input key for SDIFF as the first one, the result must be empty, so we don't need to process the elements to test. This method is like the one done in zset‘s `zsetChooseDiffAlgorithm` Co-authored-by:
Oran Agra <oran@redislabs.com>
-
wutao_water authored
update macros ZIPLIST_ENTRY_END i think the right definition is ((zl)+intrev32ifbe(ZIPLIST_BYTES(zl))-ZIPLIST_END_SIZE)
-
David CARLIER authored
Till now, on MacOS we only used to enable SO_KEEPALIVE, but we didn't set the interval which is configurable via the `tcp-keepalive` config. This adds support for that on MacOS, to match what we already do on Linux.
-
- 01 May, 2022 1 commit
-
-
Lu JJ authored
`the redis object pointer was populated.` -> `the sds pointer was populated.` We don't populate the redis object pointer in this function.
-
- 28 Apr, 2022 3 commits
-
-
Lu JJ authored
add a comment to `container` in `quicklist.h`. Because `PLAIN` and `PACKED` are not as easy to understand as `NONE` and `LISTPACK` and we don't have a detailed comment on it. Co-authored-by:
Oran Agra <oran@redislabs.com>
-
Itamar Haber authored
-
Wen Hui authored
-
- 27 Apr, 2022 2 commits
-
-
Oran Agra authored
# Lua readonly tables The PR adds support for readonly tables on Lua to prevent security vulnerabilities: * (CVE-2022-24736) An attacker attempting to load a specially crafted Lua script can cause NULL pointer dereference which will result with a crash of the redis-server process. This issue affects all versions of Redis. * (CVE-2022-24735) By exploiting weaknesses in the Lua script execution environment, an attacker with access to Redis can inject Lua code that will execute with the (potentially higher) privileges of another Redis user. The PR is spitted into 4 commits. ### Change Lua to support readonly tables This PR modifies the Lua interpreter code to support a new flag on tables. The new flag indicating that the table is readonly and any attempt to perform any writes on such a table will result in an error. The new feature can be turned off and on using the new `lua_enablereadonlytable` Lua API. The new API can be used **only** from C code. Changes to support this feature was taken from https://luau-lang.org/ ### Change eval script to set user code on Lua registry Today, Redis wrap the user Lua code with a Lua function. For example, assuming the user code is: ``` return redis.call('ping') ``` The actual code that would have sent to the Lua interpreter was: ``` f_b3a02c833904802db9c34a3cf1292eee3246df3c() return redis.call('ping') end ``` The warped code would have been saved on the global dictionary with the following name: `f_<script sha>` (in our example `f_b3a02c833904802db9c34a3cf1292eee3246df3c`). This approach allows one user to easily override the implementation of another user code, example: ``` f_b3a02c833904802db9c34a3cf1292eee3246df3c = function() return 'hacked' end ``` Running the above code will cause `evalsha b3a02c833904802db9c34a3cf1292eee3246df3c 0` to return `hacked` although it should have returned `pong`. Another disadvantage is that Redis basically runs code on the loading (compiling) phase without been aware of it. User can do code injection like this: ``` return 1 end <run code on compling phase> function() return 1 ``` The warped code will look like this and the entire `<run code on compiling phase>` block will run outside of eval or evalsha context: ``` f_<sha>() return 1 end <run code on compling phase> function() return 1 end ``` The commits puts the user code on a special Lua table called the registry. This table is not accessible to the user so it can not be manipulated by him. Also there is no longer a need to warp the user code so there is no risk in code injection which will cause running code in the wrong context. ### Use `lua_enablereadonlytable` to protect global tables on eval and function The commit uses the new `lua_enablereadonlytable` Lua API to protect the global tables of both evals scripts and functions. For eval scripts, the implementation is easy, We simply call `lua_enablereadonlytable` on the global table to turn it into a readonly table. On functions its more complected, we want to be able to switch globals between load run and function run. To achieve this, we create a new empty table that acts as the globals table for function, we control the actual globals using metatable manipulations. Notice that even if the user gets a pointer to the original tables, all the tables are set to be readonly (using `lua_enablereadonlytable` Lua API) so he can not change them. The following better explains the solution: ``` Global table {} <- global table metatable {.__index = __real_globals__} ``` The `__real_globals__` is depends on the run context (function load or function call). Why is this solution needed and its not enough to simply switch globals? When we run in the context of function load and create our functions, our function gets the current globals that was set when they were created. Replacing the globals after the creation will not effect them. This is why this trick it mandatory. ### Protect the rest of the global API and add an allowed list to the provided API The allowed list is done by setting a metatable on the global table before initialising any library. The metatable set the `__newindex` field to a function that check the allowed list before adding the field to the table. Fields which is not on the allowed list are simply ignored. After initialisation phase is done we protect the global table and each table that might be reachable from the global table. For each table we also protect the table metatable if exists. ### Performance Performance tests was done on a private computer and its only purpose is to show that this fix is not causing any performance regression. case 1: `return redis.call('ping')` case 2: `for i=1,10000000 do redis.call('ping') end` | | Unstable eval | Unstable function | lua_readonly_tables eval | lua_readonly_tables function | |-----------------------------|---------------|-------------------|--------------------------|------------------------------| | case1 ops/sec | 235904.70 | 236406.62 | 232180.16 | 230574.14 | | case1 avg latency ms | 0.175 | 0.164 | 0.178 | 0.149 | | case2 total time in seconds | 3.373 | 3.444s | 3.268 | 3.278 | ### Breaking changes * `print` function was removed from Lua because it can potentially cause the Redis processes to get stuck (if no one reads from stdout). Users should use redis.log. An alternative is to override the `print` implementation and print the message to the log file. All the work by @MeirShpilraien, i'm just publishing it.
-
chenyang8094 authored
Followup fix for #10616
-