- 29 Sep, 2024 1 commit
-
-
Moti Cohen authored
This PR introduces a new `SFLUSH` command to cluster mode that allows partial flushing of nodes based on specified slot ranges. Current implementation is designed to flush all slots of a shard, but future extensions could allow for more granular flushing. **Command Usage:** `SFLUSH <start-slot> <end-slot> [<start-slot> <end-slot>]* [SYNC|ASYNC]` This command removes all data from the specified slots, either synchronously or asynchronously depending on the optional SYNC/ASYNC argument. **Functionality:** Current imp of `SFLUSH` command verifies that the provided slot ranges are valid and cover all of the node's slots before proceeding. If slots are partially or incorrectly specified, the command will fail and return an error, ensuring that all slots of a node must be fully covered for the flush to proceed. The function supports both synchronous (default) and asynchronous flushing. In addition, if possible, SFLUSH SYNC will be run as blocking ASYNC as an optimization.
-
- 02 Apr, 2024 1 commit
-
-
Moti Cohen authored
# Overview Users utilize the `FLUSHDB SYNC` and `FLUSHALL SYNC` commands for a variety of reasons. The main issue with this command is that if the database becomes substantial in size, the server will be unresponsive for an extended period. Other than freezing application traffic, this may also lead some clients making incorrect judgments about the server's availability. For instance, a watchdog may erroneously decide to terminate the process, resulting in potential adverse outcomes. While a `FLUSH* ASYNC` can address these issues, it might not be used for two reasons: firstly, it's not the default, and secondly, in some cases, the client issuing the flush wants to wait for its completion before repopulating the database. Between the option of triggering FLUSH* asynchronously in the background without indication for completion versus running it synchronously in the foreground by the main thread, there is another more appealing option. We can block the client that requested the flush, execute the flush command in the background, and once done, unblock the client and return notification for completion. This approach ensures the server remains responsive to other clients, and the blocked client receives the expected response only after the flush operation has been successfully carried out. # Implementation details Instead of defining yet another flavor to the flush command, we can modify `FLUSHALL SYNC` and `FLUSHDB SYNC` always run in this new mode. ## Extending BIO Threads capabilities Today jobs that are carried out by BIO threads don't have the capability to indicate completion to the main thread. We can add this infrastructure by having an additional dummy job, coined as completion-job, that eventually will be written by BIO threads to a response-queue. The main thread will take care to consume items from the response-queue and call the provided callback function of each completion-job. ## FLUSH* SYNC to run as blocking ASYNC Command `FLUSH* SYNC` will be modified to create one or more async jobs to flush DB(s) and afterward will push additional completion-job request. By sending the completion job request only at the end, the main thread will be called back only after all the preceding jobs completed their task in the background. During that time, the client of the command is suspended and marked as `BLOCKED_LAZYFREE` whereas any other client will be able to communicate with the server without any issue.
-
- 20 Mar, 2024 1 commit
-
-
Pieter Cailliau authored
[Read more about the license change here](https://redis.com/blog/redis-adopts-dual-source-available-licensing/) Live long and prosper
🖖
-
- 21 Jun, 2023 1 commit
-
-
Binbin authored
Now we can see something like this: ``` Fatal: Can't initialize Background Jobs. Error message: Cannot allocate memory ```
-
- 14 Mar, 2023 1 commit
-
-
Slava Koyfman authored
Implementing the WAITAOF functionality which would allow the user to block until a specified number of Redises have fsynced all previous write commands to the AOF. Syntax: `WAITAOF <num_local> <num_replicas> <timeout>` Response: Array containing two elements: num_local, num_replicas num_local is always either 0 or 1 representing the local AOF on the master. num_replicas is the number of replicas that acknowledged the a replication offset of the last write being fsynced to the AOF. Returns an error when called on replicas, or when called with non-zero num_local on a master with AOF disabled, in all other cases the response just contains number of fsync copies. Main changes: * Added code to keep track of replication offsets that are confirmed to have been fsynced to disk. * Keep advancing master_repl_offset even when replication is disabled (and there's no replication backlog, only if there's an AOF enabled). This way we can use this command and it's mechanisms even when replication is disabled. * Extend REPLCONF ACK to `REPLCONF ACK <ofs> FACK <ofs>`, the FACK will be appended only if there's an AOF on the replica, and already ignored on old masters (thus backwards compatible) * WAIT now no longer wait for the replication offset after your last command, but rather the replication offset after your last write (or read command that caused propagation, e.g. lazy expiry). Unrelated changes: * WAIT command respects CLIENT_DENY_BLOCKING (not just CLIENT_MULTI) Implementation details: * Add an atomic var named `fsynced_reploff_pending` that's updated (usually by the bio thread) and later copied to the main `fsynced_reploff` variable (only if the AOF base file exists). I.e. during the initial AOF rewrite it will not be used as the fsynced offset since the AOF base is still missing. * Replace close+fsync bio job with new BIO_CLOSE_AOF (AOF specific) job that will also update fsync offset the field. * Handle all AOF jobs (BIO_CLOSE_AOF, BIO_AOF_FSYNC) in the same bio worker thread, to impose ordering on their execution. This solves a race condition where a job could set `fsynced_reploff_pending` to a higher value than another pending fsync job, resulting in indicating an offset for which parts of the data have not yet actually been fsynced. Imposing an ordering on the jobs guarantees that fsync jobs are executed in increasing order of replication offset. * Drain bio jobs when switching `appendfsync` to "always" This should prevent a write race between updates to `fsynced_reploff_pending` in the main thread (`flushAppendOnlyFile` when set to ALWAYS fsync), and those done in the bio thread. * Drain the pending fsync when starting over a new AOF to avoid race conditions with the previous AOF offsets overriding the new one (e.g. after switching to replicate from a new master). * Make sure to update the fsynced offset at the end of the initial AOF rewrite. a must in case there are no additional writes that trigger a periodic fsync, specifically for a replica that does a full sync. Limitations: It is possible to write a module and a Lua script that propagate to the AOF and doesn't propagate to the replication stream. see REDISMODULE_ARGV_NO_REPLICAS and luaRedisSetReplCommand. These features are incompatible with the WAITAOF command, and can result in two bad cases. The scenario is that the user executes command that only propagates to AOF, and then immediately issues a WAITAOF, and there's no further writes on the replication stream after that. 1. if the the last thing that happened on the replication stream is a PING (which increased the replication offset but won't trigger an fsync on the replica), then the client would hang forever (will wait for an fack that the replica will never send sine it doesn't trigger any fsyncs). 2. if the last thing that happened is a write command that got propagated properly, then WAITAOF will be released immediately, without waiting for an fsync (since the offset didn't change) Refactoring: * Plumbing to allow bio worker to handle multiple job types This introduces infrastructure necessary to allow BIO workers to not have a 1-1 mapping of worker to job-type. This allows in the future to assign multiple job types to a single worker, either as a performance/resource optimization, or as a way of enforcing ordering between specific classes of jobs. Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 12 Feb, 2023 1 commit
-
-
Tian authored
# Background The RDB file is usually generated and used once and seldom used again, but the content would reside in page cache until OS evicts it. A potential problem is that once the free memory exhausts, the OS have to reclaim some memory from page cache or swap anonymous page out, which may result in a jitters to the Redis service. Supposing an exact scenario, a high-capacity machine hosts many redis instances, and we're upgrading the Redis together. The page cache in host machine increases as RDBs are generated. Once the free memory drop into low watermark(which is more likely to happen in older Linux kernel like 3.10, before [watermark_scale_factor](https://lore.kernel.org/lkml/1455813719-2395-1-git-send-email-hannes@cmpxchg.org/) is introduced, the `low watermark` is linear to `min watermark`, and there'is not too much buffer space for `kswapd` to be wake up to reclaim memory), a `direct reclaim` happens, which means the process would stall to wait for memory allocation. # What the PR does The PR introduces a capability to reclaim the cache when the RDB is operated. Generally there're two cases, read and write the RDB. For read it's a little messy to address the incremental reclaim, so the reclaim is done in one go in background after the load is finished to avoid blocking the work thread. For write, incremental reclaim amortizes the work of reclaim so no need to put it into background, and the peak watermark of cache can be reduced in this way. Two cases are addresses specially, replication and restart, for both of which the cache is leveraged to speed up the processing, so the reclaim is postponed to a right time. To do this, a flag is added to`rdbSave` and `rdbLoad` to control whether the cache need to be kept, with the default value false. # Something deserve noting 1. Though `posix_fadvise` is the POSIX standard, but only few platform support it, e.g. Linux, FreeBSD 10.0. 2. In Linux `posix_fadvise` only take effect on writeback-ed pages, so a `sync`(or `fsync`, `fdatasync`) is needed to flush the dirty page before `posix_fadvise` if we reclaim write cache. # About test A unit test is added to verify the effect of `posix_fadvise`. In integration test overall cache increase is checked, as well as the cache backed by RDB as a specific TCL test is executed in isolated Github action job.
-
- 26 Aug, 2022 1 commit
-
-
Moti Cohen authored
* Remove redundant array bio_pending[]. Value at index i identically reflects the length of list bio_jobs[i]. Better use listLength() instead and discard this array. (no critical section issues to concern about). changed returned value of bioPendingJobsOfType() from "long long" to "long". Remove unused API. Maybe we will use this API later.
-
- 25 Jul, 2022 1 commit
-
-
Binbin authored
In rewriteAppendOnlyFileBackground, after flushAppendOnlyFile(1), and before openNewIncrAofForAppend, we should call redis_fsync to fsync the aof file. Because we may open a new INCR AOF in openNewIncrAofForAppend, in the case of using everysec policy, the old AOF file may not be fsynced in time (or even at all). When using everysec, we don't want to pay the disk latency from the main thread, so we will do a background fsync. Adding a argument for bioCreateCloseJob, a `need_fsync` flag to indicate that a fsync is required before the file is closed. So we will fsync the old AOF file before we close it. A cleanup, we make union become a union, since the free_* args and the fd / fsync args are never used together. Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 10 Oct, 2021 1 commit
-
-
menwen authored
looks like this field was never actually used and the call to time() is excessive.
-
- 11 Apr, 2021 1 commit
-
-
Wang Yuan authored
The bio aof fsync fd may be closed by main thread (AOFRW done handler) and even possibly reused for another socket, pipe, or file. This can can an EBADF or EINVAL fsync error, which will lead to -MISCONF errors failing all writes. We just ignore these errno because aof fsync did not really fail. We handle errno when fsyncing aof in bio, so we could know the real reason when users get -MISCONF Errors writing to the AOF file error Issue created with #8419
-
- 01 Apr, 2021 1 commit
-
-
Wang Yuan authored
In `aof.c`, we call fsync when stop aof, and now print a log to let user know that if fail. In `cluster.c`, we now return error, the calling function already handles these write errors. In `redis-cli.c`, users hope to save rdb, we now print a message if fsync failed. In `rio.c`, we now treat fsync errors like we do for write errors. In `server.c`, we try to fsync aof file when shutdown redis, we only can print one log if fail. In `bio.c`, if failing to fsync aof file, we will set `aof_bio_fsync_status` to error , and reject writing just like last writing aof error, moreover also set INFO command field `aof_last_write_status` to error.
-
- 24 Dec, 2020 1 commit
-
-
Madelyn Olson authored
Cleanup key tracking documentation, always cleanup the tracking table, and free the tracking table in an async manner when applicable.
-
- 02 Dec, 2020 1 commit
-
-
Wang Yuan authored
Backup keys to slots map and restore when fail to sync if diskless-load type is swapdb in cluster mode (#8108) When replica diskless-load type is swapdb in cluster mode, we didn't backup keys to slots map, so we will lose keys to slots map if fail to sync. Now we backup keys to slots map at first, and restore it properly when fail. This commit includes a refactory/cleanup of the backups mechanism (moving it to db.c and re-structuring it a bit). Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 21 Sep, 2020 1 commit
-
-
WuYunlong authored
Refine comment of makeThreadKillable(). This commit can be backported to 5.0, only if we also backport 8b70cb0e . Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 16 Sep, 2020 1 commit
-
-
WuYunlong authored
If one thread got SIGSEGV, function sigsegvHandler() would be triggered, it would call bioKillThreads(). But call pthread_cancel() to cancel itself would make it block. Also note that if SIGSEGV is caught by bio thread, it should kill the main thread in order to give a positive report.
-
- 11 Aug, 2020 1 commit
-
-
Muhammad Zahalqa authored
Fix unidentical function declaration in bio.c. lazyfree.c: lazyfreeFreeSlotsMapFromBioThread (#7228)
-
- 02 May, 2020 1 commit
-
-
zhenwei pi authored
Currently, there are several types of threads/child processes of a redis server. Sometimes we need deeply optimise the performance of redis, so we would like to isolate threads/processes. There were some discussion about cpu affinity cases in the issue: https://github.com/antirez/redis/issues/2863 So implement cpu affinity setting by redis.conf in this patch, then we can config server_cpulist/bio_cpulist/aof_rewrite_cpulist/ bgsave_cpulist by cpu list. Examples of cpulist in redis.conf: server_cpulist 0-7:2 means cpu affinity 0,2,4,6 bio_cpulist 1,3 means cpu affinity 1,3 aof_rewrite_cpulist 8-11 means cpu affinity 8,9,10,11 bgsave_cpulist 1,10-11 means cpu affinity 1,10,11 Test on linux/freebsd, both work fine. Signed-off-by:
zhenwei pi <pizhenwei@bytedance.com>
-
- 20 Apr, 2020 1 commit
-
-
Dave-in-lafayette authored
If there's a panic before all threads have been started (say, if file descriptor 0 is closed at exec), the panic response will crash here again.
-
- 18 Apr, 2020 1 commit
-
-
zhenwei pi authored
Set thread name for each thread of redis-server, this helps us to monitor the utilization and optimise the performance. And suggested-by Salvatore, implement this feature for multi platforms. Currently support linux and bsd, ignore other OS. An exmaple on Linux: # top -d 5 -p `pidof redis-server ` -H PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3682671 root 20 0 227744 8248 3836 R 99.2 0.0 0:19.53 redis-server 3682677 root 20 0 227744 8248 3836 S 26.4 0.0 0:04.15 io_thd_3 3682675 root 20 0 227744 8248 3836 S 23.6 0.0 0:03.98 io_thd_1 3682676 root 20 0 227744 8248 3836 S 23.6 0.0 0:03.97 io_thd_2 3682672 root 20 0 227744 8248 3836 S 0.2 0.0 0:00.02 bio_close_file 3682673 root 20 0 227744 8248 3836 S 0.2 0.0 0:00.02 bio_aof_fsync 3682674 root 20 0 227744 8248 3836 S 0.0 0.0 0:00.00 bio_lazy_free 3682678 root 20 0 227744 8248 3836 S 0.0 0.0 0:00.00 jemalloc_bg_thd 3682682 root 20 0 227744 8248 3836 S 0.0 0.0 0:00.00 jemalloc_bg_thd 3682683 root 20 0 227744 8248 3836 S 0.0 0.0 0:00.00 jemalloc_bg_thd 3682684 root 20 0 227744 8248 3836 S 0.0 0.0 0:00.00 jemalloc_bg_thd 3682685 root 20 0 227744 8248 3836 S 0.0 0.0 0:00.00 jemalloc_bg_thd 3682687 root 20 0 227744 8248 3836 S 0.0 0.0 0:00.00 jemalloc_bg_thd Another exmaple on FreeBSD-12.1: PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 5212 root 100 0 48M 7280K CPU2 2 0:26 99.52% redis-server{redis-server} 5212 root 38 0 48M 7280K umtxn 4 0:06 26.94% redis-server{io_thd_3} 5212 root 36 0 48M 7280K umtxn 6 0:06 26.84% redis-server{io_thd_1} 5212 root 39 0 48M 7280K umtxn 1 0:06 25.30% redis-server{io_thd_2} 5212 root 20 0 48M 7280K uwait 3 0:00 0.00% redis-server{redis-server} 5212 root 21 0 48M 7280K uwait 2 0:00 0.00% redis-server{bio_close_file} 5212 root 21 0 48M 7280K uwait 3 0:00 0.00% redis-server{bio_aof_fsync} 5212 root 21 0 48M 7280K uwait 0 0:00 0.00% redis-server{bio_lazy_free} Signed-off-by:
zhenwei pi <pizhenwei@bytedance.com>
-
- 05 Oct, 2018 1 commit
-
-
antirez authored
-
- 05 Sep, 2018 1 commit
-
-
youjiali1995 authored
-
- 15 Mar, 2018 1 commit
-
-
zhaozhao.zz authored
-
- 01 Oct, 2015 4 commits
- 27 Jul, 2015 2 commits
- 26 Jul, 2015 2 commits
- 30 Mar, 2015 1 commit
-
-
antirez authored
Another one just to avoid a warning. Slightly more defensive code anyway.
-
- 19 Jan, 2013 1 commit
-
-
guiquanz authored
-
- 22 Nov, 2012 1 commit
-
-
antirez authored
We use this new bio.c feature in order to stop our I/O threads if there is a memory test to do on crash. In this case we don't want anything else than the main thread to run, otherwise the other threads may mess with the heap and the memory test will report a false positive.
-
- 08 Nov, 2012 1 commit
-
-
antirez authored
-
- 27 Mar, 2012 1 commit
-
-
antirez authored
This is required to ensure that the signal will be delivered to the main thread when the watchdog timer expires.
-
- 19 Sep, 2011 1 commit
-
-
antirez authored
-
- 16 Sep, 2011 1 commit
-
-
antirez authored
-
- 15 Sep, 2011 3 commits