- 18 Jan, 2024 5 commits
-
-
Chen Tianjie authored
When doing dict resizing, dictTypeResizeAllowed is used to judge whether the new allocated memory for rehashing would cause OOM. However when shrinking, we alloc `_dictNextExp(d->ht_used[0])` bytes of memory, while in `dictTypeResizeAllowed` we still use `_dictNextExp(d->ht_used[0]+1)` as the new allocated memory size. This will overestimate the memory used by shrinking at special conditions, causing a false OOM judgement.
-
Binbin authored
Introduced in #12952, reported by valgrind.
-
Binbin authored
We doing this in diskless on-empty-db mode, when diskless loading fails, we will call emptyData to remove the half-loaded data in case we started with an empty replica. Now when a disk-based sync rdbLoad fails, we will call emptyData too in case it loads partially incomplete data. when the replica attempts another re-sync, it'll empty the dataset again anyway, so this affects two things: 1. memory consumption in the time gap until the next rdb loading begins 2. if the unsynced replica is for some reason promoted, it would have kept the partial dataset instead of being empty.
-
Binbin authored
Before #12850, we will only try to shrink the dict in serverCron, which we can control by using a child process, but now every time we delete a key, the shrink check will be called. In these test (added in #12802), we meant to disable the resizing, but druing the delete, the dict will meet the force shrink, like 2 / 128 = 0.015 < 0.2, the delete will trigger a force resize and will cause the test to fail. In this commit, we try to keep the load factor at 3 / 128 = 0.023, that is, do not meet the force shrink.
-
Binbin authored
In the past we used integers to compare ratios, let us assume that we have the following data in expanding: ``` used / size > 5 `80 / 16 > 5` is false `81 / 16 > 5` is false `95 / 16 > 5` is false `96 / 16 > 5` is true ``` Because the integer result is rounded, our resize breaks the ratio constraint, this has existed since the beginning, which resulted in us not strictly following the ratio (shrink also has the same issue). This PR change it to multiplication to avoid floating point calculations.
-
- 17 Jan, 2024 1 commit
-
-
Binbin authored
The test have a race: ``` *** [err]: Redis can rewind and trigger smaller slot resizing in tests/unit/other.tcl Expected '[Dictionary HT] Hash table 0 stats (main hash table): table size: 12 number of elements: 2 [Expires HT] Hash table 0 stats (main hash table): No stats available for empty dictionaries ' to match '*table size: 8*' (context: type eval line 12 cmd {assert_match "*table size: 8*" [r debug HTSTATS 0]} proc ::test) ``` When `r del "{alice}$j"` is executed in the loop, when the key is deleted to [9, 12], the load factor has meet HASHTABLE_MIN_FILL, if serverCron happens to trigger slot dict resize, then the test will fail. Because there is not way to meet HASHTABLE_MIN_FILL in the subsequent dels. The solution is to avoid triggering the resize in advance. We can use multi to delete them at once, or we can disable the resize. Since we disabled resize in the previous test, the fix also uses the method of disabling resize. The test is introduced in #12802.
-
- 15 Jan, 2024 3 commits
-
-
Binbin authored
The new shrink was added in #12850. Also updated outdated comments, see #11692.
-
Yanqi Lv authored
When we insert entries into dict, it may autonomously expand if needed. However, when we delete entries from dict, it doesn't shrink to the proper size. If there are few entries in a very large dict, it may cause huge waste of memory and inefficiency when iterating. The main keyspace dicts (keys and expires), are shrinked by cron (`tryResizeHashTables` calls `htNeedsResize` and `dictResize`), And some data structures such as zset and hash also do that (call `htNeedsResize`) right after a loop of calls to `dictDelete`, But many other dicts are completely missing that call (they can only expand). In this PR, we provide the ability to automatically shrink the dict when deleting. The conditions triggering the shrinking is the same as `htNeedsResize` used to have. i.e. we expand when we're over 100% utilization, and shrink when we're below 10% utilization. Additionally: * Add `dictPauseAutoResize` so that flows that do mass deletions, will only trigger shrinkage at the end. * Rename `dictResize` to `dictShrinkToFit` (same logic as it used to have, but better name describing it) * Rename `_dictExpand` to `_dictResize` (same logic as it used to have, but better name describing it) related to discussion https://github.com/redis/redis/pull/12819#discussion_r1409293878 --------- Co-authored-by:
Oran Agra <oran@redislabs.com> Co-authored-by:
zhaozhao.zz <zhaozhao.zz@alibaba-inc.com>
-
zhaozhao.zz authored
Regarding how to obtain the hash slot of a key, there is an optimization in `getKeySlot()`, it is used to avoid redundant hash calculations for keys: when the current client is in the process of executing a command, it can directly use the slot of the current client because the slot to access has already been calculated in advance in `processCommand()`. However, scripts are a special case where, in default mode or with `allow-cross-slot-keys` enabled, they are allowed to access keys beyond the pre-declared range. This means that the keys they operate on may not belong to the slot of the pre-declared keys. Currently, when the commands in a script are executed, the slot of the original client (i.e., the current client) is not correctly updated, leading to subsequent access to the wrong slot. This PR fixes the above issue. When checking the cluster constraints in a script, the slot to be accessed by the current command is set for the original client (i.e., the current client). This ensures that `getKeySlot()` gets the correct slot cache. Additionally, the following modifications are made: 1. The 'sort' and 'sort_ro' commands use `getKeySlot()` instead of `c->slot` because the client could be an engine client in a script and can lead to potential bug. 2. `getKeySlot()` is also used in pubsub to obtain the slot for the channel, standardizing the way slots are retrieved.
-
- 14 Jan, 2024 1 commit
-
-
Binbin authored
The open function returns a fd on success or -1 on failure, here we should check fd != -1, otherwise -1 will be judged as success. This closes #12938.
-
- 12 Jan, 2024 1 commit
-
-
Chen Tianjie authored
Change the calculation method of bytes_per_key to make it closer to the true average key size. The calculation method is as follows: mh->bytes_per_key = mh->total_keys ? (mh->dataset / mh->total_keys) : 0;
-
- 11 Jan, 2024 2 commits
-
-
Harkrishn Patro authored
Avoid crash while performing `DEBUG CLUSTERLINK KILL` mutliple times (cluster link might not be created/valid).
-
bentotten authored
When one shard, sole primary node marks potentially failed replica as FAIL instead of PFAIL (#12824) Fixes issue where a single primary cannot mark a replica as failed in a single-shard cluster.
-
- 10 Jan, 2024 1 commit
-
-
Binbin authored
The test was introduced in #10745, but we forgot to add it to the test_helper.tcl, so our CI did not actually run it. This PR adds it and ensures it passes CI tests.
-
- 09 Jan, 2024 2 commits
-
-
Oran Agra authored
#11766 introduced a bug in sdsResize where it could forget to update the sds type in the sds header and then cause an overflow in sdsalloc. it looks like the only implication of that is a possible assertion in HLL, but it's hard to rule out possible heap corruption issues with clientsCronResizeQueryBuffer
-
Madelyn Olson authored
Fix a daily test failure because alpine doesn't support stack traces and add in an extra assertion related to making sure the stack trace was printed twice.
-
- 08 Jan, 2024 5 commits
-
-
Binbin authored
We should close server.rdb_child_exit_pipe when redisFork fails, otherwise the pipe fd will be leaked. Just a cleanup.
-
Andy Pan authored
This is a follow-up PR for #12782, in which we introduced nested preprocessor directives for TCP keep-alive on Solaris and added redundant indentation for code. Besides, it could result in unreachable code due to the lack of `#else` on the latest Solaris 11.4 where `TCP_KEEPIDLE`, `TCP_KEEPINTVL`, and `TCP_KEEPCNT` are available. As a result, this PR does three main things: - To eliminate the redundant indention for C code in nested preprocessor directives - To add `#else` directives and move `TCP_KEEPALIVE_THRESHOLD` + `TCP_KEEPALIVE_ABORT_THRESHOLD` settings under it, avoid unreachable code and compiler warnings when `#if defined(TCP_KEEPIDLE) && defined(TCP_KEEPINTVL) && defined(TCP_KEEPCNT)` is met on Solaris 11.4 - To remove a few trailing whitespace in comments
-
Yanqi Lv authored
I'm testing the performance of Pub/Sub command recently. I find if many clients unsubscribe or are killed simultaneously, Redis needs a long time to deal with it. In my experiment, I set 5000 clients and each client subscribes 100 channels. Then I call `client kill type pubsub` to simulate the situation where clients unsubscribe all channels at the same time and calculate the execution time. The result shows that it takes about 23s. I use the _perf_ and find that `listSearchKey` in `pubsubUnsubscribeChannel` costs more than 90% cpu time. I think we can optimize this situation. In this PR, I replace list with dict to track the clients subscribing the channel more efficiently. It changes O(N) to O(1) in the search phase. Then I repeat the experiment as above. The results are as follows. | | Execution Time(s) |used_memory(MB) | | :---------------- | :------: | :----: | | unstable(1bd0b549) | 23.734 | 65.41 | | optimize-pubsub | 0.288 | 67.66 | Thanks for #11595 , I use a no-value dict and the results shows that the performance improves significantly but the memory usage only increases slightly. Notice: - This PR will cause the performance degradation about 20% in `[p|s]subscribe` command but won't freeze Redis.
-
debing.sun authored
In #10122, we set the destination key's flag of SINTERSTORE to `RW`, however, this command doesn't actually read or modify the destination key, just overwrites it. Therefore, we change it to `OW` similarly to all other *STORE commands.
-
Binbin authored
Crash reported in #12695. In the process of upgrading the cluster from 7.0 to 7.2, because the 7.0 nodes will not gossip shard id, in 7.2 we will rely on shard id to build the server.cluster->shards dict. In some cases, for example, the 7.0 master node and the 7.2 replica node. From the view of 7.2 replica node, the cluster->shards dictionary does not have its master node. In this case calling CLUSTER SHARDS on the 7.2 replica node may crash. We should fix the underlying assumption of updateShardId, which is that the shard dict should be always in sync with the node's shard_id. The fix was suggested by PingXie, see more details in #12695.
-
- 07 Jan, 2024 2 commits
-
-
debing.sun authored
## Issues and solutions from #12817 1. Touch ProcessingEventsWhileBlocked and calling moduleCount() without GIL in afterSleep() - Introduced: Version: 7.0.0 PR: #9963 - Harm Level: Very High If the module thread calls `RM_Yield()` before the main thread enters afterSleep(), and modifies `ProcessingEventsWhileBlocked`(+1), it will cause the main thread to not wait for GIL, which can lead to all kinds of unforeseen problems, including memory data corruption. - Initial / Abandoned Solution: * Added `__thread` specifier for ProcessingEventsWhileBlocked. `ProcessingEventsWhileBlocked` is used to protect against nested event processing, but event processing in the main thread and module threads should be completely independent and unaffected, so it is safer to use TLS. * Adding a cached module count to keep track of the current number of modules, to avoid having to use `dictSize()`. - Related Warnings: ``` WARNING: ThreadSanitizer: data race (pid=1136) Write of size 4 at 0x0001045990c0 by thread T4 (mutexes: write M0): #0 processEventsWhileBlocked networking.c:4135 (redis-server:arm64+0x10006d124) #1 RM_Yield module.c:2410 (redis-server:arm64+0x10018b66c) #2 bg_call_worker <null>:83232836 (blockedclient.so:arm64+0x16a8) Previous read of size 4 at 0x0001045990c0 by main thread: #0 afterSleep server.c:1861 (redis-server:arm64+0x100024f98) #1 aeProcessEvents ae.c:408 (redis-server:arm64+0x10000fd64) #2 aeMain ae.c:496 (redis-server:arm64+0x100010f0c) #3 main server.c:7220 (redis-server:arm64+0x10003f38c) ``` 2. aeApiPoll() is not thread-safe When using RM_Yield to handle events in a module thread, if the main thread has not yet entered `afterSleep()`, both the module thread and the main thread may touch `server.el` at the same time. - Introduced: Version: 7.0.0 PR: #9963 - Old / Abandoned Solution: Adding a new mutex to protect timing between after beforeSleep() and before afterSleep(). Defect: If the main thread enters the ae loop without any IO events, it will wait until the next timeout or until there is any event again, and the module thread will always hang until the main thread leaves the event loop. - Related Warnings: ``` SUMMARY: ThreadSanitizer: data race ae_kqueue.c:55 in addEventMask ================== ================== WARNING: ThreadSanitizer: data race (pid=14682) Write of size 4 at 0x000100b54000 by thread T9 (mutexes: write M0): #0 aeApiPoll ae_kqueue.c:175 (redis-server:arm64+0x100010588) #1 aeProcessEvents ae.c:399 (redis-server:arm64+0x10000fb84) #2 processEventsWhileBlocked networking.c:4138 (redis-server:arm64+0x10006d3c4) #3 RM_Yield module.c:2410 (redis-server:arm64+0x10018b66c) #4 bg_call_worker <null>:16042052 (blockedclient.so:arm64+0x169c) Previous write of size 4 at 0x000100b54000 by main thread: #0 aeApiPoll ae_kqueue.c:175 (redis-server:arm64+0x100010588) #1 aeProcessEvents ae.c:399 (redis-server:arm64+0x10000fb84) #2 aeMain ae.c:496 (redis-server:arm64+0x100010da8) #3 main server.c:7238 (redis-server:arm64+0x10003f51c) ``` ## The final fix as the comments: https://github.com/redis/redis/pull/12817#discussion_r1436427232 Optimized solution based on the above comment: First, we add `module_gil_acquring` to indicate whether the main thread is currently in the acquiring GIL state. When the module thread starts to yield, there are two possibilities(we assume the caller keeps the GIL): 1. The main thread is in the mid of beforeSleep() and afterSleep(), that is, `module_gil_acquring` is not 1 now. At this point, the module thread will wake up the main thread through the pipe and leave the yield, waiting for the next yield when the main thread may already in the acquiring GIL state. 2. The main thread is in the acquiring GIL state. The module thread release the GIL, yielding CPU to give the main thread an opportunity to start event processing, and then acquire the GIL again until the main thread releases it. This is what https://github.com/redis/redis/pull/12817#discussion_r1436427232 mentioned direction. --------- Co-authored-by:
Oran Agra <oran@redislabs.com>
-
Binbin authored
If there are nodes in the cluster that do not support shard-id, they will gossip shard-id. From the perspective of nodes that support shard-id, their shard-id is meaningless (since shard-id is randomly generated when we create a node.) Nodes that support shard-id will save the shard-id information in nodes.conf. If the node is restarted according to nodes.conf, the server will report a corrupted cluster config file error. Because auxShardIdSetter will reject configurations with inconsistent master-replica shard-ids. A cluster-wide consensus for the node's shard_id is not necessary. The key is maintaining consistency of the shard_id on each individual 7.2 node. As the cluster progressively upgrades to version 7.2, we can expect the shard_ids across all nodes to naturally converge and align. In this PR, when processing the gossip, if sender is a replica and does not support shard-id, set the shard_id to the shard_id of its master.
-
- 04 Jan, 2024 1 commit
-
-
dependabot[bot] authored
Bumps [cross-platform-actions/action](https://github.com/cross-platform-actions/action) from 0.21.1 to 0.22.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/cross-platform-actions/action/releases">cross-platform-actions/action's releases</a>.</em></p> <blockquote> <h2>Cross Platform Action 0.22.0</h2> <h3>Added</h3> <ul> <li> <p>Added support for using the action in multiple steps in the same job (<a href="https://redirect.github.com/cross-platform-actions/action/issues/26">#26</a>). All the inputs need to be the same for all steps, except for the following inputs: <code>sync_files</code>, <code>shutdown_vm</code> and <code>run</code>.</p> </li> <li> <p>Added support for specifying that the VM should not shutdown after the action has run. This adds a new input parameter: <code>shutdown_vm</code>. When set to <code>false</code>, this will hopefully mitigate very frequent freezing of VM during teardown (<a href="https://redirect.github.com/cross-platform-actions/action/issues/61">#61</a>, <a href="https://redirect.github.com/cross-platform-actions/action/issues/72">#72</a>).</p> </li> </ul> <h3>Changed</h3> <ul> <li> <p>Always terminate VM instead of shutting down. This is more efficient and this will hopefully mitigate very frequent freezing of VM during teardown (<a href="https://redirect.github.com/cross-platform-actions/action/issues/61">#61</a>, <a href="https://redirect.github.com/cross-platform-actions/action/issues/72">#72</a>).</p> </li> <li> <p>Use <code>unsafe</code> as the cache mode for QEMU disks. This should improve performance (<a href="https://redirect.github.com/cross-platform-actions/action/issues/67">#67</a>).</p> </li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/cross-platform-actions/action/blob/master/changelog.md">cross-platform-actions/action's changelog</a>.</em></p> <blockquote> <h2>[0.22.0] - 2023-12-27</h2> <h3>Added</h3> <ul> <li> <p>Added support for using the action in multiple steps in the same job (<a href="https://redirect.github.com/cross-platform-actions/action/issues/26">#26</a>). All the inputs need to be the same for all steps, except for the following inputs: <code>sync_files</code>, <code>shutdown_vm</code> and <code>run</code>.</p> </li> <li> <p>Added support for specifying that the VM should not shutdown after the action has run. This adds a new input parameter: <code>shutdown_vm</code>. When set to <code>false</code>, this will hopefully mitigate very frequent freezing of VM during teardown (<a href="https://redirect.github.com/cross-platform-actions/action/issues/61">#61</a>, <a href="https://redirect.github.com/cross-platform-actions/action/issues/72">#72</a>).</p> </li> </ul> <h3>Changed</h3> <ul> <li> <p>Always terminate VM instead of shutting down. This is more efficient and this will hopefully mitigate very frequent freezing of VM during teardown (<a href="https://redirect.github.com/cross-platform-actions/action/issues/61">#61</a>, <a href="https://redirect.github.com/cross-platform-actions/action/issues/72">#72</a>).</p> </li> <li> <p>Use <code>unsafe</code> as the cache mode for QEMU disks. This should improve performance (<a href="https://redirect.github.com/cross-platform-actions/action/issues/67">#67</a>).</p> </li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/cross-platform-actions/action/commit/5800fa0060a22edf69992a779adac3d2bb3a6f8a"><code>5800fa0</code></a> Release 0.22.0</li> <li><a href="https://github.com/cross-platform-actions/action/commit/20ad4b2cebd9f7e4d0b9a17e71a811178b95e698"><code>20ad4b2</code></a> Fix <a href="https://redirect.github.com/cross-platform-actions/action/issues/67">#67</a>: Use <code>unsafe</code> as the cache mode disks</li> <li><a href="https://github.com/cross-platform-actions/action/commit/d9184930c333266f6819290a62817f3dc884ffd5"><code>d918493</code></a> Always terminate VM instead of shutting down.</li> <li><a href="https://github.com/cross-platform-actions/action/commit/626f1d6c958e17e60619b6a2936726ed4c59a36f"><code>626f1d6</code></a> Fix error when terminating the VM</li> <li><a href="https://github.com/cross-platform-actions/action/commit/d59f08dc5c932d21a2663daac30c3a4942a177ba"><code>d59f08d</code></a> Print stack trace for uncaught exceptions</li> <li><a href="https://github.com/cross-platform-actions/action/commit/7f2fab9c5601e3f6f057283dbac16f581ded8a8b"><code>7f2fab9</code></a> Revert "Run SSH in verbose mode when debug mode is enabled"</li> <li><a href="https://github.com/cross-platform-actions/action/commit/0f566c356e98c968c75937a2c38a5de468ffc0bb"><code>0f566c3</code></a> [no ci] Update the changelog</li> <li><a href="https://github.com/cross-platform-actions/action/commit/b7f77446bb2277f3e4c16ddd30056c8bc1b706f7"><code>b7f7744</code></a> [no ci] Fix spelling</li> <li><a href="https://github.com/cross-platform-actions/action/commit/9894a9b11876c7fc7872617db592776585446d8f"><code>9894a9b</code></a> Wrap <code>host</code> module in namespace</li> <li><a href="https://github.com/cross-platform-actions/action/commit/87fdd346a2f9d849e6cd66745590059eb3e52fe2"><code>87fdd34</code></a> Fix broken test-vm-shutdown tests</li> <li>Additional commits viewable in <a href="https://github.com/cross-platform-actions/action/compare/v0.21.1...v0.22.0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores ) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by:
dependabot[bot] <support@github.com> Co-authored-by:
dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
-
- 03 Jan, 2024 2 commits
-
-
Lior Kogan authored
- Referring to Redis Discord channel instead of the mailing list. - Referring to the licensing instead of repeating it.
-
Madelyn Olson authored
This change is trying to make two failure modes a bit easier to deep dive: 1. If a serverPanic or serverAssert occurs during the info (or module) printing, it will recursively panic, which is a lot of fun as it will just keep recursively printing. It will eventually stack overflow, but will generate a lot of text in the process. 2. When a segfault happens during the segfault handler, no information is communicated other than it happened. This can be problematic because `info` may help diagnose the real issue, but without fixing the recursive crash it might be hard to get at that info.
-
- 02 Jan, 2024 1 commit
-
-
AshMosh authored
There are situations (especially in TLS) in which the engine gets too occupied managing a large number of new connections. Existing connections may time-out while the server is processing the new connections initial TLS handshakes, which may cause cause new connections to be established, perpetuating the problem. To better manage the tradeoff between new connection rate and other workloads, this change adds a new config to manage maximum number of new connections per event loop cycle, instead of using a predetermined number (currently 1000). This change introduces two new configurations, max-new-connections-per-cycle and max-new-tls-connections-per-cycle. The default value of the tcp connections is 10 per cycle and the default value of tls connections per cycle is 1. --------- Co-authored-by:
Madelyn Olson <madelyneolson@gmail.com>
-
- 28 Dec, 2023 5 commits
-
-
Chen Tianjie authored
This PR is a supplement to #11144, moving `signalModifiedKey` in `xaddCommand` after the trimming, to ensure the state of key eventual consistency. Currently there is no problem with Redis, but it is better to avoid issues in future development on Redis.
-
dependabot[bot] authored
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2 to 3. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/github/codeql-action/releases">github/codeql-action's releases</a>.</em></p> <blockquote> <h2>CodeQL Bundle v2.15.4</h2> <p>Bundles CodeQL CLI v2.15.4</p> <ul> <li>(<a href="https://github.com/github/codeql-cli-binaries/blob/HEAD/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql-cli-binaries/releases/tag/v2.15.4">release</a>)</li> </ul> <p>Includes the following CodeQL language packs from <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4"><code>github/codeql@codeql-cli/v2.15.4</code></a>:</p> <ul> <li><code>codeql/cpp-queries</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/cpp/ql/src/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/cpp/ql/src">source</a>)</li> <li><code>codeql/cpp-all</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/cpp/ql/lib/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/cpp/ql/lib">source</a>)</li> <li><code>codeql/csharp-queries</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/csharp/ql/src/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/csharp/ql/src">source</a>)</li> <li><code>codeql/csharp-all</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/csharp/ql/lib/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/csharp/ql/lib">source</a>)</li> <li><code>codeql/go-queries</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/go/ql/src/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/go/ql/src">source</a>)</li> <li><code>codeql/go-all</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/go/ql/lib/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/go/ql/lib">source</a>)</li> <li><code>codeql/java-queries</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/java/ql/src/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/java/ql/src">source</a>)</li> <li><code>codeql/java-all</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/java/ql/lib/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/java/ql/lib">source</a>)</li> <li><code>codeql/javascript-queries</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/javascript/ql/src/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/javascript/ql/src">source</a>)</li> <li><code>codeql/javascript-all</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/javascript/ql/lib/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/javascript/ql/lib">source</a>)</li> <li><code>codeql/python-queries</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/python/ql/src/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/python/ql/src">source</a>)</li> <li><code>codeql/python-all</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/python/ql/lib/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/python/ql/lib">source</a>)</li> <li><code>codeql/ruby-queries</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/ruby/ql/src/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/ruby/ql/src">source</a>)</li> <li><code>codeql/ruby-all</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/ruby/ql/lib/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/ruby/ql/lib">source</a>)</li> <li><code>codeql/swift-queries</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/swift/ql/src/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/swift/ql/src">source</a>)</li> <li><code>codeql/swift-all</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/swift/ql/lib/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/swift/ql/lib">source</a>)</li> </ul> <h2>CodeQL Bundle</h2> <p>Bundles CodeQL CLI v2.15.3</p> <ul> <li>(<a href="https://github.com/github/codeql-cli-binaries/blob/HEAD/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql-cli-binaries/releases/tag/v2.15.3">release</a>)</li> </ul> <p>Includes the following CodeQL language packs from <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3"><code>github/codeql@codeql-cli/v2.15.3</code></a>:</p> <ul> <li><code>codeql/cpp-queries</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/cpp/ql/src/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/cpp/ql/src">source</a>)</li> <li><code>codeql/cpp-all</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/cpp/ql/lib/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/cpp/ql/lib">source</a>)</li> <li><code>codeql/csharp-queries</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/csharp/ql/src/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/csharp/ql/src">source</a>)</li> <li><code>codeql/csharp-all</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/csharp/ql/lib/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/csharp/ql/lib">source</a>)</li> <li><code>codeql/go-queries</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/go/ql/src/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/go/ql/src">source</a>)</li> <li><code>codeql/go-all</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/go/ql/lib/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/go/ql/lib">source</a>)</li> <li><code>codeql/java-queries</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/java/ql/src/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/java/ql/src">source</a>)</li> <li><code>codeql/java-all</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/java/ql/lib/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/java/ql/lib">source</a>)</li> <li><code>codeql/javascript-queries</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/javascript/ql/src/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/javascript/ql/src">source</a>)</li> <li><code>codeql/javascript-all</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/javascript/ql/lib/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/javascript/ql/lib">source</a>)</li> <li><code>codeql/python-queries</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/python/ql/src/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/python/ql/src">source</a>)</li> <li><code>codeql/python-all</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/python/ql/lib/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/python/ql/lib">source</a>)</li> <li><code>codeql/ruby-queries</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/ruby/ql/src/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/ruby/ql/src">source</a>)</li> <li><code>codeql/ruby-all</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/ruby/ql/lib/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/ruby/ql/lib">source</a>)</li> <li><code>codeql/swift-queries</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/swift/ql/src/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/swift/ql/src">source</a>)</li> <li><code>codeql/swift-all</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/swift/ql/lib/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/swift/ql/lib">source</a>)</li> </ul> <h2>CodeQL Bundle</h2> <p>Bundles CodeQL CLI v2.15.2</p> <ul> <li>(<a href="https://github.com/github/codeql-cli-binaries/blob/HEAD/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql-cli-binaries/releases/tag/v2.15.2">release</a>)</li> </ul> <p>Includes the following CodeQL language packs from <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.2"><code>github/codeql@codeql-cli/v2.15.2</code></a>:</p> <ul> <li><code>codeql/cpp-queries</code> (<a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.2/cpp/ql/src/CHANGELOG.md">changelog</a>, <a href="https://github.com/github/codeql/tree/codeql-cli/v2.15.2/cpp/ql/src">source</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/github/codeql-action/blob/main/CHANGELOG.md">github/codeql-action's changelog</a>.</em></p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/github/codeql-action/commit/3a9f6a89e06fb331ce22481637d19617501a5b7f"><code>3a9f6a8</code></a> update javascript files</li> <li><a href="https://github.com/github/codeql-action/commit/cc4fead714532424ab15b501a01d18c7f34d17e2"><code>cc4fead</code></a> update version in various hardcoded locations</li> <li><a href="https://github.com/github/codeql-action/commit/183559cea87c163bb01e99e0a15fd5500b23307f"><code>183559c</code></a> Merge branch 'main' into update-bundle/codeql-bundle-v2.15.4</li> <li><a href="https://github.com/github/codeql-action/commit/5b52b36d41451ee775917788e3bf225d7c1ceab3"><code>5b52b36</code></a> reintroduce PR check that confirm action can be still be compiled on node16</li> <li><a href="https://github.com/github/codeql-action/commit/5b19bef41e08b00f3d8f48219f46b1866a4f940d"><code>5b19bef</code></a> change to node20 for all actions</li> <li><a href="https://github.com/github/codeql-action/commit/f2d0c2e7ae58b70f81b81bf4c8b5c7a81fb2a5d6"><code>f2d0c2e</code></a> upgrade node type definitions</li> <li><a href="https://github.com/github/codeql-action/commit/d651fbc494fab13836ab76ffcfc8ce08c7c98d78"><code>d651fbc</code></a> change to node20 for all actions</li> <li><a href="https://github.com/github/codeql-action/commit/382a50a0284c0de445104889a9d6003acb4b3c1d"><code>382a50a</code></a> Merge pull request <a href="https://redirect.github.com/github/codeql-action/issues/2021">#2021</a> from github/mergeback/v2.22.9-to-main-c0d1daa7</li> <li><a href="https://github.com/github/codeql-action/commit/458b4226ad8e38f90ff6a4ad1e18ab2593e7e3dc"><code>458b422</code></a> Update checked-in dependencies</li> <li><a href="https://github.com/github/codeql-action/commit/5e0f9dbc48f564b68392e465dcdacd74eab63e25"><code>5e0f9db</code></a> Update changelog and version after v2.22.9</li> <li>See full diff in <a href="https://github.com/github/codeql-action/compare/v2...v3">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores ) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by:
dependabot[bot] <support@github.com> Co-authored-by:
dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
-
guybe7 authored
In case server.fsynced_reploff changed (e.g. flushAppendOnly set it to server.master_repl_offset in case there was nothing to fsync) we want to avoid sleeping before the next beforeSleep so we can call blockedBeforeSleep ASAP. without that, in case there's no incoming traffic, we could be waiting for the next cron timer event to wake us up.
-
Binbin authored
The code does not delete the corresponding node when traversing clients, resulting in a loop, causing the dictDelete() == DICT_OK assertion to fail. In addition, did a cleanup, in the dictCreate scenario, we can avoid a dictFind call since the dict is empty. Issue was introduced in #12804.
-
Binbin authored
When arity is -2, it allows us to input two nodes, but returns: ``` *** ERROR: Invalid configuration for cluster creation. *** Redis Cluster requires at least 3 master nodes. *** This is not possible with 2 nodes and 0 replicas per node. *** At least 3 nodes are required. ``` When we input one node, it return: ``` [ERR] Wrong number of arguments for specified --cluster sub command ``` Strictly speaking -2 should also be rejected, because redis-cli requires at least three nodes. However, the error message was not very friendly, so decided to change it arity -1. This closes #12891.
-
- 27 Dec, 2023 3 commits
-
-
Chen Tianjie authored
We have achieved replacing `slots_to_keys` radix tree with key->slot linked list (#9356), and then replacing the list with slot specific dictionaries for keys (#11695). Shard channels behave just like keys in many ways, and we also need a slots->channels mapping. Currently this is still done by using a radix tree. So we should split `server.pubsubshard_channels` into 16384 dicts and drop the radix tree, just like what we did to DBs. Some benefits (basically the benefits of what we've done to DBs): 1. Optimize counting channels in a slot. This is currently used only in removing channels in a slot. But this is potentially more useful: sometimes we need to know how many channels there are in a specific slot when doing slot migration. Counting is now implemented by traversing the radix tree, and with this PR it will be as simple as calling `dictSize`, from O(n) to O(1). 2. The radix tree in the cluster has been removed. The shard channel names no longer require additional storage, which can save memory. 3. Potentially useful in slot migration, as shard channels are logically split by slots, thus making it easier to migrate, remove or add as a whole. 4. Avoid rehashing a big dict when there is a large number of channels. Drawbacks: 1. Takes more memory than using radix tree when there are relatively few shard channels. What this PR does: 1. in cluster mode, split `server.pubsubshard_channels` into 16384 dicts, in standalone mode, still use only one dict. 2. drop the `slots_to_channels` radix tree. 3. to save memory (to solve the drawback above), all 16384 dicts are created lazily, which means only when a channel is about to be inserted to the dict will the dict be initialized, and when all channels are deleted, the dict would delete itself. 5. use `server.shard_channel_count` to keep track of the number of all shard channels. --------- Co-authored-by:
Viktor Söderqvist <viktor.soderqvist@est.tech>
-
Moshe Kaplan authored
If fopen() is successful and redis_fstat determines that the file is 0 bytes, the file handle stored in fp will leak. This change closes the filehandle stored in fp if the file is 0 bytes. Second attempt at fixing Coverity 390029 This is a follow-up to #12796
-
sundb authored
Fix #12792 On ubuntu 23(lunar), non-root users will not be allowed to change the oom_score_adj of a process to a value that is too low. Since terminal's default oom_score_adj is 200, if we run the test on terminal, we won't be able to set the oom_score_adj of the redis process to 9 or 22, which is too low. Reproduction on ubuntu 23(lunar) terminal: ```sh $ cat /proc/`pgrep redis-server`/oom_score_adj 200 $ echo 100 > /proc/`pgrep redis-server`/oom_score_adj # success without error $ echo 99 > /proc/`pgrep redis-server`/oom_score_adj echo: write error: Permission denied ``` As from the output above, we can only set the minimum oom score of redis processes to 100. By modifying the test, make oom_score_adj only increase upwards and not decrease. --------- Co-authored-by:
debing.sun <debing.sun@redis.com>
-
- 26 Dec, 2023 4 commits
-
-
Andy Pan authored
## TCP Keep-Alives [TCP Keep-Alives](https://datatracker.ietf.org/doc/html/rfc9293#name-tcp-keep-alives) provides a way to detect whether a TCP connection is alive or dead, which can be useful for reducing system resources by cleaning up dead connections. There is full support of TCP Keep-Alives on Linux and partial support on macOS in `redis` at present. This PR intends to complete the rest. ## Unix-like OS's support `TCP_KEEPIDLE`, `TCP_KEEPINTVL`, and `TCP_KEEPCNT` are not included in the POSIX standard for `setsockopts`, while these three socket options are widely available on most Unix-like systems and Windows. ### References - [AIX](https://www.ibm.com/support/pages/ibm-aix-tcp-keepalive-probes) - [DragonflyBSD](https://man.dragonflybsd.org/?command=tcp§ion=4) - [FreeBSD](https://www.freebsd.org/cgi/man.cgi?query=tcp) - [HP-UX](https://docstore.mik.ua/manuals/hp-ux/en/B2355-60130/TCP.7P.html) - [illumos](https://illumos.org/man/4P/tcp) - [Linux](https://man7.org/linux/man-pages/man7/tcp.7.html) - [NetBSD](https://man.netbsd.org/NetBSD-8.0/tcp.4) - [Windows](https://learn.microsoft.com/en-us/windows/win32/winsock/ipproto-tcp-socket-options) ### Mac OS In earlier versions, macOS only supported setting `TCP_KEEPALIVE` (the equivalent of `TCP_KEEPIDLE` on other platforms), but since macOS 10.8 it has supported `TCP_KEEPINTVL` and `TCP_KEEPCNT`. Check out [this mailing list](https://lists.apple.com/archives/macnetworkprog/2012/Jul/msg00005.html) and [the source code](https://github.com/apple/darwin-xnu/blob/main/bsd/netinet/tcp.h#L215-L230) for more details. ### Solaris Solaris claimed it supported the TCP-Alives mechanism, but `TCP_KEEPIDLE`, `TCP_KEEPINTVL`, and `TCP_KEEPCNT` were not available on Solaris until the latest version 11.4. Therefore, we need to simulate the TCP-Alives mechanism on other platforms via `TCP_KEEPALIVE_THRESHOLD` + `TCP_KEEPALIVE_ABORT_THRESHOLD`. - [Solaris 11.3](https://docs.oracle.com/cd/E86824_01/html/E54777/tcp-7p.html) - [Solaris 11.4](https://docs.oracle.com/cd/E88353_01/html/E37851/tcp-4p.html ) --------- Co-authored-by:
Oran Agra <oran@redislabs.com>
-
Jeff Liu authored
add a missing comment for `dont_compress` and fix the bits calculation
-
zalj authored
The implementation of aeProcessEvents seems have different behavior from the top comment. The implementation process file events first, then process time events.
-
Binbin authored
When testing and debugging the cluster code before, you need to stop the cluster after making changes, and then start the cluster again. Add a restart option for ease of use.
-
- 24 Dec, 2023 1 commit
-
-
Slava Koyfman authored
Previous implementation would disconnect _all_ clients when running `ACL LOAD`, which wasn't very useful. This change brings the behavior in line with that of `ACL SETUSER`, `ACL DELUSER`, in that only clients whose user is deleted or clients subscribed to channels which they no longer have access to will be disconnected. --------- Co-authored-by:
Oran Agra <oran@redislabs.com> Co-authored-by:
Madelyn Olson <34459052+madolson@users.noreply.github.com>
-