Unverified Commit ec2d1807 authored by Oran Agra's avatar Oran Agra Committed by GitHub
Browse files

Merge 6.2 RC2

parents b8c67ce4 049f2f08
================================================================================
Redis 6.2 RC2 Released Tue Jan 12 16:17:20 IST 2021
================================================================================
Upgrade urgency LOW: This is the second Release Candidate of Redis 6.2.
IMPORTANT: If you're running Redis on ARM64 or a big-endian system, upgrade may
have significant implications. Please be sure to read the notes below.
Here is a comprehensive list of changes in this release compared to 6.2 RC1,
each one includes the PR number that added it, so you can get more details
at https://github.com/redis/redis/pull/<number>
New commands / args:
* Add the REV, BYLEX and BYSCORE arguments to ZRANGE, and the ZRANGESTORE command (#7844)
* Add the XAUTOCLAIM command (#7973)
* Add the MINID trimming strategy and the LIMIT argument to XADD and XTRIM (#8169)
* Add the ANY argument to GEOSEARCH and GEORADIUS (#8259)
* Add the CH, NX, XX arguments to GEOADD (#8227)
* Add the COUNT argument to LPOP and RPOP (#8179)
* Add the WRITE argument to CLIENT PAUSE for pausing write commands exclusively (#8170)
* Change the proto-ver argument of HELLO to optional (#7377)
* Add the CLIENT TRACKINGINFO subcommand (#7309)
Command behavior changes:
* CLIENT TRACKING yields an error when given overlapping BCAST prefixes (#8176)
* SWAPDB invalidates WATCHed keys (#8239)
* SORT command behaves differently when used on a writable replica (#8283)
Other behavior changes:
* Avoid propagating MULTI/EXEC for read-only transactions (#8216)
* Remove the read-only flag from TIME, ECHO, ROLE, LASTSAVE (#8216)
* Fix the command flags of PFDEBUG (#8222)
* Tracking clients will no longer receive unnecessary key invalidation messages after FLUSHDB (#8039)
* Sentinel: Fix missing updates to the config file after SENTINEL SET command (#8229)
Bug fixes with compatibility implications (bugs introduced in Redis 6.0):
* Fix RDB CRC64 checksum on big-endian systems (#8270)
If you're using big-endian please consider the compatibility implications with
RESTORE, replication and persistence.
* Fix wrong order of key/value in Lua's map response (#8266)
If your scripts use redis.setresp() or return a map (new in Redis 6.0), please
consider the implications.
Bug fixes that are only applicable to previous releases of Redis 6.2:
* Resolve rare assertions in active defragmentation while loading (#8284, #8281)
Bug fixes:
* Fix the selection of a random element from large hash tables (#8133)
* Fix an issue where a forked process deletes the parent's pidfile (#8231)
* Fix crashes when enabling io-threads-do-reads (#8230)
* Fix a crash in redis-cli after executing cluster backup (#8267)
* Fix redis-benchmark to use an IP address for the first cluster node (#8154)
* Fix saving of strings larger than 2GB into RDB files (#8306)
Additional improvements:
* Improve replication handshake time (#8214)
* Release client tracking table memory asynchronously in cases where the DB is also freed asynchronously (#8039)
* Avoid wasteful transient memory allocation in certain cases (#8286, #5954)
* Handle binary string values by the 'requirepass' and 'masterauth' configs (#8200)
Platform and deployment-related changes:
* Install redis-check-rdb and redis-check-aof as symlinks to redis-server (#5745)
* Add a check for an ARM64 Linux kernel bug (#8224)
Due to the potential severity of this issue, Redis will refuse to run on
affected platforms by default.
Info fields and introspection changes:
* Add the errorstats section to the INFO command (#8217)
* Add the failed_calls and rejected_calls fields INFO's commandstats section (#8217)
* Report child copy-on-write metrics continuously (#8264)
Module API changes:
* Add the RedisModule_SendChildCOWInfo API (#8264)
* Add the may-replicate command flag (#8170)
================================================================================ ================================================================================
Redis 6.2 RC1 Released Mon Dec 14 11:50:00 IST 2020 Redis 6.2 RC1 Released Mon Dec 14 11:50:00 IST 2020
================================================================================ ================================================================================
Upgrade urgency LOW: This is the first RC of Redis 6.2. Upgrade urgency LOW: This is the first Release Candidate of Redis 6.2.
Introduction to the Redis 6.2 release Introduction to the Redis 6.2 release
===================================== =====================================
...@@ -29,7 +105,7 @@ New commands / args: ...@@ -29,7 +105,7 @@ New commands / args:
* Add SMISMEMBER command that checks multiple members (#7615) * Add SMISMEMBER command that checks multiple members (#7615)
* Add ZMSCORE command that returns an array of scores (#7593) * Add ZMSCORE command that returns an array of scores (#7593)
* Add LMOVE and BLMOVE commands that pop and push arbitrarily (#6929) * Add LMOVE and BLMOVE commands that pop and push arbitrarily (#6929)
* Add RESET command that resets client connection state(#7982) * Add RESET command that resets client connection state (#7982)
* Add COPY command that copies keys (#7953) * Add COPY command that copies keys (#7953)
* Add ZDIFF and ZDIFFSTORE commands (#7961) * Add ZDIFF and ZDIFFSTORE commands (#7961)
* Add ZINTER and ZUNION commands (#7794) * Add ZINTER and ZUNION commands (#7794)
...@@ -146,7 +222,7 @@ Platform / toolchain support related improvements: ...@@ -146,7 +222,7 @@ Platform / toolchain support related improvements:
New configuration options: New configuration options:
* Enable configuring OpenSSL using the standard openssl.cnf (#8143) * Enable configuring OpenSSL using the standard openssl.cnf (#8143)
* oom-score-adj-values can config can now take absolute values (besides relative ones) (#8046) * oom-score-adj-values config can now take absolute values (besides relative ones) (#8046)
* TLS: Add different client cert support. (#8076) * TLS: Add different client cert support. (#8076)
* Note that a few other changes listed above added their config options. * Note that a few other changes listed above added their config options.
...@@ -177,31 +253,42 @@ We'll follow up with more RC releases, until the code looks production ready ...@@ -177,31 +253,42 @@ We'll follow up with more RC releases, until the code looks production ready
and we don't get reports of serious issues for a while. and we don't get reports of serious issues for a while.
A special thank you for the amount of work put into this release by: A special thank you for the amount of work put into this release by:
- Oran Agra
- Yossi Gottlieb
- Itamar Haber
- Guy Benoish
- Filipe Oliveira
- Madelyn Olson
- Wang Yuan - Wang Yuan
- Felipe Machado - Felipe Machado
- Filipe Oliveira
- Tatsuya Arisawa
- Yang Bodong - Yang Bodong
- Tatsuya Arisawa
- Jonah H. Harris
- Jim Brunner - Jim Brunner
- Yaacov Hazan
- Wen Hui - Wen Hui
- Guy Benoish
- Chen Yang - Chen Yang
- Nitai Caro - Nitai Caro
- Meir Shpilraien - Meir Shpilraien
- Madelyn Olson
- maohuazhu - maohuazhu
- Valentino Geron - Valentino Geron
- Qu Chen
- sundb
- George Prekas
- Zhao Zhao
- Tyson Andre - Tyson Andre
- Michael Grunder - Michael Grunder
- alexronke-channeladvisor - alexronke-channeladvisor
- sundb - Wu Yunlong
- WuYunlong - Wei Kukey
- Yoav Steinberg - Yoav Steinberg
- Uri Shachar - Uri Shachar
- Greg Femec
- Nykolas Laurentino de Lima - Nykolas Laurentino de Lima
- xhe
- zhenwei pi
- David CARLIER - David CARLIER
Migrating from 6.0 to 6.2 Migrating from 6.0 to 6.2
========================= =========================
......
...@@ -51,7 +51,7 @@ ...@@ -51,7 +51,7 @@
#define _EL_CLEANUP(ctx) do { \ #define _EL_CLEANUP(ctx) do { \
if ((ctx)->ev.cleanup) (ctx)->ev.cleanup((ctx)->ev.data); \ if ((ctx)->ev.cleanup) (ctx)->ev.cleanup((ctx)->ev.data); \
ctx->ev.cleanup = NULL; \ ctx->ev.cleanup = NULL; \
} while(0); } while(0)
static inline void refreshTimeout(redisAsyncContext *ctx) { static inline void refreshTimeout(redisAsyncContext *ctx) {
#define REDIS_TIMER_ISSET(tvp) \ #define REDIS_TIMER_ISSET(tvp) \
......
...@@ -196,9 +196,12 @@ tcp-keepalive 300 ...@@ -196,9 +196,12 @@ tcp-keepalive 300
# #
# tls-cluster yes # tls-cluster yes
# Explicitly specify TLS versions to support. Allowed values are case insensitive # By default, only TLSv1.2 and TLSv1.3 are enabled and it is highly recommended
# and include "TLSv1", "TLSv1.1", "TLSv1.2", "TLSv1.3" (OpenSSL >= 1.1.1) or # that older formally deprecated versions are kept disabled to reduce the attack surface.
# any combination. To enable only TLSv1.2 and TLSv1.3, use: # You can explicitly specify TLS versions to support.
# Allowed values are case insensitive and include "TLSv1", "TLSv1.1", "TLSv1.2",
# "TLSv1.3" (OpenSSL >= 1.1.1) or any combination.
# To enable only TLSv1.2 and TLSv1.3, use:
# #
# tls-protocols "TLSv1.2 TLSv1.3" # tls-protocols "TLSv1.2 TLSv1.3"
...@@ -688,7 +691,7 @@ replica-priority 100 ...@@ -688,7 +691,7 @@ replica-priority 100
# Redis implements server assisted support for client side caching of values. # Redis implements server assisted support for client side caching of values.
# This is implemented using an invalidation table that remembers, using # This is implemented using an invalidation table that remembers, using
# 16 millions of slots, what clients may have certain subsets of keys. In turn # a radix key indexed by key name, what clients have which keys. In turn
# this is used in order to send invalidation messages to clients. Please # this is used in order to send invalidation messages to clients. Please
# check this page to understand more about the feature: # check this page to understand more about the feature:
# #
...@@ -1973,3 +1976,10 @@ jemalloc-bg-thread yes ...@@ -1973,3 +1976,10 @@ jemalloc-bg-thread yes
# #
# Set bgsave child process to cpu affinity 1,10,11 # Set bgsave child process to cpu affinity 1,10,11
# bgsave_cpulist 1,10-11 # bgsave_cpulist 1,10-11
# In some cases redis will emit warnings and even refuse to start if it detects
# that the system is in bad state, it is possible to suppress these warnings
# by setting the following config which takes a space delimited list of warnings
# to suppress
#
# ignore-warnings ARM64-COW-BUG
...@@ -115,8 +115,18 @@ else ...@@ -115,8 +115,18 @@ else
ifeq ($(uname_S),Darwin) ifeq ($(uname_S),Darwin)
# Darwin # Darwin
FINAL_LIBS+= -ldl FINAL_LIBS+= -ldl
# Homebrew's OpenSSL is not linked to /usr/local to avoid
# conflicts with the system's LibreSSL installation so it
# must be referenced explicitly during build.
ifeq ($(uname_M),arm64)
# Homebrew arm64 uses /opt/homebrew as HOMEBREW_PREFIX
OPENSSL_CFLAGS=-I/opt/homebrew/opt/openssl/include
OPENSSL_LDFLAGS=-L/opt/homebrew/opt/openssl/lib
else
# Homebrew x86/ppc uses /usr/local as HOMEBREW_PREFIX
OPENSSL_CFLAGS=-I/usr/local/opt/openssl/include OPENSSL_CFLAGS=-I/usr/local/opt/openssl/include
OPENSSL_LDFLAGS=-L/usr/local/opt/openssl/lib OPENSSL_LDFLAGS=-L/usr/local/opt/openssl/lib
endif
else else
ifeq ($(uname_S),AIX) ifeq ($(uname_S),AIX)
# AIX # AIX
...@@ -260,11 +270,11 @@ endif ...@@ -260,11 +270,11 @@ endif
REDIS_SERVER_NAME=redis-server$(PROG_SUFFIX) REDIS_SERVER_NAME=redis-server$(PROG_SUFFIX)
REDIS_SENTINEL_NAME=redis-sentinel$(PROG_SUFFIX) REDIS_SENTINEL_NAME=redis-sentinel$(PROG_SUFFIX)
REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crcspeed.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o lolwut.o lolwut5.o lolwut6.o acl.o gopher.o tracking.o connection.o tls.o sha256.o timeout.o setcpuaffinity.o monotonic.o REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crcspeed.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o lolwut.o lolwut5.o lolwut6.o acl.o gopher.o tracking.o connection.o tls.o sha256.o timeout.o setcpuaffinity.o monotonic.o mt19937-64.o
REDIS_CLI_NAME=redis-cli$(PROG_SUFFIX) REDIS_CLI_NAME=redis-cli$(PROG_SUFFIX)
REDIS_CLI_OBJ=anet.o adlist.o dict.o redis-cli.o zmalloc.o release.o ae.o crcspeed.o crc64.o siphash.o crc16.o monotonic.o cli_common.o REDIS_CLI_OBJ=anet.o adlist.o dict.o redis-cli.o zmalloc.o release.o ae.o crcspeed.o crc64.o siphash.o crc16.o monotonic.o cli_common.o mt19937-64.o
REDIS_BENCHMARK_NAME=redis-benchmark$(PROG_SUFFIX) REDIS_BENCHMARK_NAME=redis-benchmark$(PROG_SUFFIX)
REDIS_BENCHMARK_OBJ=ae.o anet.o redis-benchmark.o adlist.o dict.o zmalloc.o release.o crcspeed.o crc64.o siphash.o crc16.o monotonic.o cli_common.o REDIS_BENCHMARK_OBJ=ae.o anet.o redis-benchmark.o adlist.o dict.o zmalloc.o release.o crcspeed.o crc64.o siphash.o crc16.o monotonic.o cli_common.o mt19937-64.o
REDIS_CHECK_RDB_NAME=redis-check-rdb$(PROG_SUFFIX) REDIS_CHECK_RDB_NAME=redis-check-rdb$(PROG_SUFFIX)
REDIS_CHECK_AOF_NAME=redis-check-aof$(PROG_SUFFIX) REDIS_CHECK_AOF_NAME=redis-check-aof$(PROG_SUFFIX)
...@@ -336,7 +346,7 @@ $(REDIS_CLI_NAME): $(REDIS_CLI_OBJ) ...@@ -336,7 +346,7 @@ $(REDIS_CLI_NAME): $(REDIS_CLI_OBJ)
$(REDIS_BENCHMARK_NAME): $(REDIS_BENCHMARK_OBJ) $(REDIS_BENCHMARK_NAME): $(REDIS_BENCHMARK_OBJ)
$(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a ../deps/hdr_histogram/hdr_histogram.o $(FINAL_LIBS) $(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a ../deps/hdr_histogram/hdr_histogram.o $(FINAL_LIBS)
dict-benchmark: dict.c zmalloc.c sds.c siphash.c dict-benchmark: dict.c zmalloc.c sds.c siphash.c mt19937-64.c
$(REDIS_CC) $(FINAL_CFLAGS) $^ -D DICT_BENCHMARK_MAIN -o $@ $(FINAL_LIBS) $(REDIS_CC) $(FINAL_CFLAGS) $^ -D DICT_BENCHMARK_MAIN -o $@ $(FINAL_LIBS)
DEP = $(REDIS_SERVER_OBJ:%.o=%.d) $(REDIS_CLI_OBJ:%.o=%.d) $(REDIS_BENCHMARK_OBJ:%.o=%.d) DEP = $(REDIS_SERVER_OBJ:%.o=%.d) $(REDIS_CLI_OBJ:%.o=%.d) $(REDIS_BENCHMARK_OBJ:%.o=%.d)
...@@ -405,8 +415,8 @@ install: all ...@@ -405,8 +415,8 @@ install: all
$(REDIS_INSTALL) $(REDIS_SERVER_NAME) $(INSTALL_BIN) $(REDIS_INSTALL) $(REDIS_SERVER_NAME) $(INSTALL_BIN)
$(REDIS_INSTALL) $(REDIS_BENCHMARK_NAME) $(INSTALL_BIN) $(REDIS_INSTALL) $(REDIS_BENCHMARK_NAME) $(INSTALL_BIN)
$(REDIS_INSTALL) $(REDIS_CLI_NAME) $(INSTALL_BIN) $(REDIS_INSTALL) $(REDIS_CLI_NAME) $(INSTALL_BIN)
$(REDIS_INSTALL) $(REDIS_CHECK_RDB_NAME) $(INSTALL_BIN) @ln -sf $(REDIS_SERVER_NAME) $(INSTALL_BIN)/$(REDIS_CHECK_RDB_NAME)
$(REDIS_INSTALL) $(REDIS_CHECK_AOF_NAME) $(INSTALL_BIN) @ln -sf $(REDIS_SERVER_NAME) $(INSTALL_BIN)/$(REDIS_CHECK_AOF_NAME)
@ln -sf $(REDIS_SERVER_NAME) $(INSTALL_BIN)/$(REDIS_SENTINEL_NAME) @ln -sf $(REDIS_SERVER_NAME) $(INSTALL_BIN)/$(REDIS_SENTINEL_NAME)
uninstall: uninstall:
......
...@@ -174,15 +174,15 @@ sds ACLHashPassword(unsigned char *cleartext, size_t len) { ...@@ -174,15 +174,15 @@ sds ACLHashPassword(unsigned char *cleartext, size_t len) {
return sdsnewlen(hex,HASH_PASSWORD_LEN); return sdsnewlen(hex,HASH_PASSWORD_LEN);
} }
/* Given a hash and the hash length, returns C_OK if it is a valid password /* Given a hash and the hash length, returns C_OK if it is a valid password
* hash, or C_ERR otherwise. */ * hash, or C_ERR otherwise. */
int ACLCheckPasswordHash(unsigned char *hash, int hashlen) { int ACLCheckPasswordHash(unsigned char *hash, int hashlen) {
if (hashlen != HASH_PASSWORD_LEN) { if (hashlen != HASH_PASSWORD_LEN) {
return C_ERR; return C_ERR;
} }
/* Password hashes can only be characters that represent /* Password hashes can only be characters that represent
* hexadecimal values, which are numbers and lowercase * hexadecimal values, which are numbers and lowercase
* characters 'a' through 'f'. */ * characters 'a' through 'f'. */
for(int i = 0; i < HASH_PASSWORD_LEN; i++) { for(int i = 0; i < HASH_PASSWORD_LEN; i++) {
char c = hash[i]; char c = hash[i];
...@@ -2184,18 +2184,30 @@ void aclCommand(client *c) { ...@@ -2184,18 +2184,30 @@ void aclCommand(client *c) {
} }
} else if (c->argc == 2 && !strcasecmp(sub,"help")) { } else if (c->argc == 2 && !strcasecmp(sub,"help")) {
const char *help[] = { const char *help[] = {
"LOAD -- Reload users from the ACL file.", "CAT [<category>]",
"SAVE -- Save the current config to the ACL file.", " List all commands that belong to <category>, or all command categories",
"LIST -- Show user details in config file format.", " when no category is specified.",
"USERS -- List all the registered usernames.", "DELUSER <username> [<username> ...]",
"SETUSER <username> [attribs ...] -- Create or modify a user.", " Delete a list of users.",
"GETUSER <username> -- Get the user details.", "GETUSER <username>",
"DELUSER <username> [...] -- Delete a list of users.", " Get the user's details.",
"CAT -- List available categories.", "GENPASS [<bits>]",
"CAT <category> -- List commands inside category.", " Generate a secure 256-bit user password. The optional `bits` argument can",
"GENPASS [<bits>] -- Generate a secure user password.", " be used to specify a different size.",
"WHOAMI -- Return the current connection username.", "LIST",
"LOG [<count> | RESET] -- Show the ACL log entries.", " Show users details in config file format.",
"LOAD",
" Reload users from the ACL file.",
"LOG [<count> | RESET]",
" Show the ACL log entries.",
"SAVE",
" Save the current config to the ACL file.",
"SETUSER <username> <attribute> [<attribute> ...]",
" Create or modify a user with the specified attributes.",
"USERS",
" List all the registered usernames.",
"WHOAMI",
" Return the current connection username.",
NULL NULL
}; };
addReplyHelp(c,help); addReplyHelp(c,help);
...@@ -2224,7 +2236,7 @@ void addReplyCommandCategories(client *c, struct redisCommand *cmd) { ...@@ -2224,7 +2236,7 @@ void addReplyCommandCategories(client *c, struct redisCommand *cmd) {
void authCommand(client *c) { void authCommand(client *c) {
/* Only two or three argument forms are allowed. */ /* Only two or three argument forms are allowed. */
if (c->argc > 3) { if (c->argc > 3) {
addReply(c,shared.syntaxerr); addReplyErrorObject(c,shared.syntaxerr);
return; return;
} }
......
...@@ -206,29 +206,27 @@ int aofFsyncInProgress(void) { ...@@ -206,29 +206,27 @@ int aofFsyncInProgress(void) {
/* Starts a background task that performs fsync() against the specified /* Starts a background task that performs fsync() against the specified
* file descriptor (the one of the AOF file) in another thread. */ * file descriptor (the one of the AOF file) in another thread. */
void aof_background_fsync(int fd) { void aof_background_fsync(int fd) {
bioCreateBackgroundJob(BIO_AOF_FSYNC,(void*)(long)fd,NULL,NULL); bioCreateFsyncJob(fd);
} }
/* Kills an AOFRW child process if exists */ /* Kills an AOFRW child process if exists */
void killAppendOnlyChild(void) { void killAppendOnlyChild(void) {
int statloc; int statloc;
/* No AOFRW child? return. */ /* No AOFRW child? return. */
if (server.aof_child_pid == -1) return; if (server.child_type != CHILD_TYPE_AOF) return;
/* Kill AOFRW child, wait for child exit. */ /* Kill AOFRW child, wait for child exit. */
serverLog(LL_NOTICE,"Killing running AOF rewrite child: %ld", serverLog(LL_NOTICE,"Killing running AOF rewrite child: %ld",
(long) server.aof_child_pid); (long) server.child_pid);
if (kill(server.aof_child_pid,SIGUSR1) != -1) { if (kill(server.child_pid,SIGUSR1) != -1) {
while(wait3(&statloc,0,NULL) != server.aof_child_pid); while(wait3(&statloc,0,NULL) != server.child_pid);
} }
/* Reset the buffer accumulating changes while the child saves. */ /* Reset the buffer accumulating changes while the child saves. */
aofRewriteBufferReset(); aofRewriteBufferReset();
aofRemoveTempFile(server.aof_child_pid); aofRemoveTempFile(server.child_pid);
server.aof_child_pid = -1; resetChildState();
server.aof_rewrite_time_start = -1; server.aof_rewrite_time_start = -1;
/* Close pipes used for IPC between the two processes. */ /* Close pipes used for IPC between the two processes. */
aofClosePipes(); aofClosePipes();
closeChildInfoPipe();
updateDictResizePolicy();
} }
/* Called when the user switches from "appendonly yes" to "appendonly no" /* Called when the user switches from "appendonly yes" to "appendonly no"
...@@ -265,14 +263,14 @@ int startAppendOnly(void) { ...@@ -265,14 +263,14 @@ int startAppendOnly(void) {
strerror(errno)); strerror(errno));
return C_ERR; return C_ERR;
} }
if (hasActiveChildProcess() && server.aof_child_pid == -1) { if (hasActiveChildProcess() && server.child_type != CHILD_TYPE_AOF) {
server.aof_rewrite_scheduled = 1; server.aof_rewrite_scheduled = 1;
serverLog(LL_WARNING,"AOF was enabled but there is already another background operation. An AOF background was scheduled to start when possible."); serverLog(LL_WARNING,"AOF was enabled but there is already another background operation. An AOF background was scheduled to start when possible.");
} else { } else {
/* If there is a pending AOF rewrite, we need to switch it off and /* If there is a pending AOF rewrite, we need to switch it off and
* start a new one: the old one cannot be reused because it is not * start a new one: the old one cannot be reused because it is not
* accumulating the AOF buffer. */ * accumulating the AOF buffer. */
if (server.aof_child_pid != -1) { if (server.child_type == CHILD_TYPE_AOF) {
serverLog(LL_WARNING,"AOF was enabled but there is already an AOF rewriting in background. Stopping background AOF and starting a rewrite now."); serverLog(LL_WARNING,"AOF was enabled but there is already an AOF rewriting in background. Stopping background AOF and starting a rewrite now.");
killAppendOnlyChild(); killAppendOnlyChild();
} }
...@@ -646,7 +644,7 @@ void feedAppendOnlyFile(struct redisCommand *cmd, int dictid, robj **argv, int a ...@@ -646,7 +644,7 @@ void feedAppendOnlyFile(struct redisCommand *cmd, int dictid, robj **argv, int a
* accumulate the differences between the child DB and the current one * accumulate the differences between the child DB and the current one
* in a buffer, so that when the child process will do its work we * in a buffer, so that when the child process will do its work we
* can append the differences to the new append only file. */ * can append the differences to the new append only file. */
if (server.aof_child_pid != -1) if (server.child_type == CHILD_TYPE_AOF)
aofRewriteBufferAppend((unsigned char*)buf,sdslen(buf)); aofRewriteBufferAppend((unsigned char*)buf,sdslen(buf));
sdsfree(buf); sdsfree(buf);
...@@ -1427,6 +1425,8 @@ int rewriteAppendOnlyFileRio(rio *aof) { ...@@ -1427,6 +1425,8 @@ int rewriteAppendOnlyFileRio(rio *aof) {
dictEntry *de; dictEntry *de;
size_t processed = 0; size_t processed = 0;
int j; int j;
long key_count = 0;
long long cow_updated_time = 0;
for (j = 0; j < server.dbnum; j++) { for (j = 0; j < server.dbnum; j++) {
char selectcmd[] = "*2\r\n$6\r\nSELECT\r\n"; char selectcmd[] = "*2\r\n$6\r\nSELECT\r\n";
...@@ -1486,6 +1486,19 @@ int rewriteAppendOnlyFileRio(rio *aof) { ...@@ -1486,6 +1486,19 @@ int rewriteAppendOnlyFileRio(rio *aof) {
processed = aof->processed_bytes; processed = aof->processed_bytes;
aofReadDiffFromParent(); aofReadDiffFromParent();
} }
/* Update COW info every 1 second (approximately).
* in order to avoid calling mstime() on each iteration, we will
* check the diff every 1024 keys */
if ((key_count & 1023) == 0) {
key_count = 0;
long long now = mstime();
if (now - cow_updated_time >= 1000) {
sendChildCOWInfo(CHILD_TYPE_AOF, 0, "AOF rewrite");
cow_updated_time = now;
}
}
key_count++;
} }
dictReleaseIterator(di); dictReleaseIterator(di);
di = NULL; di = NULL;
...@@ -1579,8 +1592,31 @@ int rewriteAppendOnlyFile(char *filename) { ...@@ -1579,8 +1592,31 @@ int rewriteAppendOnlyFile(char *filename) {
serverLog(LL_NOTICE, serverLog(LL_NOTICE,
"Concatenating %.2f MB of AOF diff received from parent.", "Concatenating %.2f MB of AOF diff received from parent.",
(double) sdslen(server.aof_child_diff) / (1024*1024)); (double) sdslen(server.aof_child_diff) / (1024*1024));
if (rioWrite(&aof,server.aof_child_diff,sdslen(server.aof_child_diff)) == 0)
goto werr; /* Now we write the entire AOF buffer we received from the parent
* via the pipe during the life of this fork child.
* once a second, we'll take a break and send updated COW info to the parent */
size_t bytes_to_write = sdslen(server.aof_child_diff);
const char *buf = server.aof_child_diff;
long long cow_updated_time = mstime();
while (bytes_to_write) {
/* We write the AOF buffer in chunk of 8MB so that we can check the time in between them */
size_t chunk_size = bytes_to_write < (8<<20) ? bytes_to_write : (8<<20);
if (rioWrite(&aof,buf,chunk_size) == 0)
goto werr;
bytes_to_write -= chunk_size;
buf += chunk_size;
/* Update COW info */
long long now = mstime();
if (now - cow_updated_time >= 1000) {
sendChildCOWInfo(CHILD_TYPE_AOF, 0, "AOF rewrite");
cow_updated_time = now;
}
}
/* Make sure data will not remain on the OS's output buffers */ /* Make sure data will not remain on the OS's output buffers */
if (fflush(fp)) goto werr; if (fflush(fp)) goto werr;
...@@ -1703,7 +1739,6 @@ int rewriteAppendOnlyFileBackground(void) { ...@@ -1703,7 +1739,6 @@ int rewriteAppendOnlyFileBackground(void) {
if (hasActiveChildProcess()) return C_ERR; if (hasActiveChildProcess()) return C_ERR;
if (aofCreatePipes() != C_OK) return C_ERR; if (aofCreatePipes() != C_OK) return C_ERR;
openChildInfoPipe();
if ((childpid = redisFork(CHILD_TYPE_AOF)) == 0) { if ((childpid = redisFork(CHILD_TYPE_AOF)) == 0) {
char tmpfile[256]; char tmpfile[256];
...@@ -1712,7 +1747,7 @@ int rewriteAppendOnlyFileBackground(void) { ...@@ -1712,7 +1747,7 @@ int rewriteAppendOnlyFileBackground(void) {
redisSetCpuAffinity(server.aof_rewrite_cpulist); redisSetCpuAffinity(server.aof_rewrite_cpulist);
snprintf(tmpfile,256,"temp-rewriteaof-bg-%d.aof", (int) getpid()); snprintf(tmpfile,256,"temp-rewriteaof-bg-%d.aof", (int) getpid());
if (rewriteAppendOnlyFile(tmpfile) == C_OK) { if (rewriteAppendOnlyFile(tmpfile) == C_OK) {
sendChildCOWInfo(CHILD_TYPE_AOF, "AOF rewrite"); sendChildCOWInfo(CHILD_TYPE_AOF, 1, "AOF rewrite");
exitFromChild(0); exitFromChild(0);
} else { } else {
exitFromChild(1); exitFromChild(1);
...@@ -1720,7 +1755,6 @@ int rewriteAppendOnlyFileBackground(void) { ...@@ -1720,7 +1755,6 @@ int rewriteAppendOnlyFileBackground(void) {
} else { } else {
/* Parent */ /* Parent */
if (childpid == -1) { if (childpid == -1) {
closeChildInfoPipe();
serverLog(LL_WARNING, serverLog(LL_WARNING,
"Can't rewrite append only file in background: fork: %s", "Can't rewrite append only file in background: fork: %s",
strerror(errno)); strerror(errno));
...@@ -1731,8 +1765,7 @@ int rewriteAppendOnlyFileBackground(void) { ...@@ -1731,8 +1765,7 @@ int rewriteAppendOnlyFileBackground(void) {
"Background append only file rewriting started by pid %ld",(long) childpid); "Background append only file rewriting started by pid %ld",(long) childpid);
server.aof_rewrite_scheduled = 0; server.aof_rewrite_scheduled = 0;
server.aof_rewrite_time_start = time(NULL); server.aof_rewrite_time_start = time(NULL);
server.aof_child_pid = childpid;
updateDictResizePolicy();
/* We set appendseldb to -1 in order to force the next call to the /* We set appendseldb to -1 in order to force the next call to the
* feedAppendOnlyFile() to issue a SELECT command, so the differences * feedAppendOnlyFile() to issue a SELECT command, so the differences
* accumulated by the parent into server.aof_rewrite_buf will start * accumulated by the parent into server.aof_rewrite_buf will start
...@@ -1745,7 +1778,7 @@ int rewriteAppendOnlyFileBackground(void) { ...@@ -1745,7 +1778,7 @@ int rewriteAppendOnlyFileBackground(void) {
} }
void bgrewriteaofCommand(client *c) { void bgrewriteaofCommand(client *c) {
if (server.aof_child_pid != -1) { if (server.child_type == CHILD_TYPE_AOF) {
addReplyError(c,"Background append only file rewriting already in progress"); addReplyError(c,"Background append only file rewriting already in progress");
} else if (hasActiveChildProcess()) { } else if (hasActiveChildProcess()) {
server.aof_rewrite_scheduled = 1; server.aof_rewrite_scheduled = 1;
...@@ -1803,7 +1836,7 @@ void backgroundRewriteDoneHandler(int exitcode, int bysignal) { ...@@ -1803,7 +1836,7 @@ void backgroundRewriteDoneHandler(int exitcode, int bysignal) {
* rewritten AOF. */ * rewritten AOF. */
latencyStartMonitor(latency); latencyStartMonitor(latency);
snprintf(tmpfile,256,"temp-rewriteaof-bg-%d.aof", snprintf(tmpfile,256,"temp-rewriteaof-bg-%d.aof",
(int)server.aof_child_pid); (int)server.child_pid);
newfd = open(tmpfile,O_WRONLY|O_APPEND); newfd = open(tmpfile,O_WRONLY|O_APPEND);
if (newfd == -1) { if (newfd == -1) {
serverLog(LL_WARNING, serverLog(LL_WARNING,
...@@ -1909,7 +1942,7 @@ void backgroundRewriteDoneHandler(int exitcode, int bysignal) { ...@@ -1909,7 +1942,7 @@ void backgroundRewriteDoneHandler(int exitcode, int bysignal) {
server.aof_state = AOF_ON; server.aof_state = AOF_ON;
/* Asynchronously close the overwritten AOF. */ /* Asynchronously close the overwritten AOF. */
if (oldfd != -1) bioCreateBackgroundJob(BIO_CLOSE_FILE,(void*)(long)oldfd,NULL,NULL); if (oldfd != -1) bioCreateCloseJob(oldfd);
serverLog(LL_VERBOSE, serverLog(LL_VERBOSE,
"Background AOF rewrite signal handler took %lldus", ustime()-now); "Background AOF rewrite signal handler took %lldus", ustime()-now);
...@@ -1931,8 +1964,7 @@ void backgroundRewriteDoneHandler(int exitcode, int bysignal) { ...@@ -1931,8 +1964,7 @@ void backgroundRewriteDoneHandler(int exitcode, int bysignal) {
cleanup: cleanup:
aofClosePipes(); aofClosePipes();
aofRewriteBufferReset(); aofRewriteBufferReset();
aofRemoveTempFile(server.aof_child_pid); aofRemoveTempFile(server.child_pid);
server.aof_child_pid = -1;
server.aof_rewrite_time_last = time(NULL)-server.aof_rewrite_time_start; server.aof_rewrite_time_last = time(NULL)-server.aof_rewrite_time_start;
server.aof_rewrite_time_start = -1; server.aof_rewrite_time_start = -1;
/* Schedule a new rewrite if we are waiting for it to switch the AOF ON. */ /* Schedule a new rewrite if we are waiting for it to switch the AOF ON. */
......
...@@ -78,15 +78,13 @@ static unsigned long long bio_pending[BIO_NUM_OPS]; ...@@ -78,15 +78,13 @@ static unsigned long long bio_pending[BIO_NUM_OPS];
* file as the API does not expose the internals at all. */ * file as the API does not expose the internals at all. */
struct bio_job { struct bio_job {
time_t time; /* Time at which the job was created. */ time_t time; /* Time at which the job was created. */
/* Job specific arguments pointers. If we need to pass more than three /* Job specific arguments.*/
* arguments we can just pass a pointer to a structure or alike. */ int fd; /* Fd for file based background jobs */
void *arg1, *arg2, *arg3; lazy_free_fn *free_fn; /* Function that will free the provided arguments */
void *free_args[]; /* List of arguments to be passed to the free function */
}; };
void *bioProcessBackgroundJobs(void *arg); void *bioProcessBackgroundJobs(void *arg);
void lazyfreeFreeObjectFromBioThread(robj *o);
void lazyfreeFreeDatabaseFromBioThread(dict *ht1, dict *ht2);
void lazyfreeFreeSlotsMapFromBioThread(rax *rt);
/* Make sure we have enough stack to perform all the things we do in the /* Make sure we have enough stack to perform all the things we do in the
* main thread. */ * main thread. */
...@@ -128,13 +126,8 @@ void bioInit(void) { ...@@ -128,13 +126,8 @@ void bioInit(void) {
} }
} }
void bioCreateBackgroundJob(int type, void *arg1, void *arg2, void *arg3) { void bioSubmitJob(int type, struct bio_job *job) {
struct bio_job *job = zmalloc(sizeof(*job));
job->time = time(NULL); job->time = time(NULL);
job->arg1 = arg1;
job->arg2 = arg2;
job->arg3 = arg3;
pthread_mutex_lock(&bio_mutex[type]); pthread_mutex_lock(&bio_mutex[type]);
listAddNodeTail(bio_jobs[type],job); listAddNodeTail(bio_jobs[type],job);
bio_pending[type]++; bio_pending[type]++;
...@@ -142,6 +135,35 @@ void bioCreateBackgroundJob(int type, void *arg1, void *arg2, void *arg3) { ...@@ -142,6 +135,35 @@ void bioCreateBackgroundJob(int type, void *arg1, void *arg2, void *arg3) {
pthread_mutex_unlock(&bio_mutex[type]); pthread_mutex_unlock(&bio_mutex[type]);
} }
void bioCreateLazyFreeJob(lazy_free_fn free_fn, int arg_count, ...) {
va_list valist;
/* Allocate memory for the job structure and all required
* arguments */
struct bio_job *job = zmalloc(sizeof(*job) + sizeof(void *) * (arg_count));
job->free_fn = free_fn;
va_start(valist, arg_count);
for (int i = 0; i < arg_count; i++) {
job->free_args[i] = va_arg(valist, void *);
}
va_end(valist);
bioSubmitJob(BIO_LAZY_FREE, job);
}
void bioCreateCloseJob(int fd) {
struct bio_job *job = zmalloc(sizeof(*job));
job->fd = fd;
bioSubmitJob(BIO_CLOSE_FILE, job);
}
void bioCreateFsyncJob(int fd) {
struct bio_job *job = zmalloc(sizeof(*job));
job->fd = fd;
bioSubmitJob(BIO_AOF_FSYNC, job);
}
void *bioProcessBackgroundJobs(void *arg) { void *bioProcessBackgroundJobs(void *arg) {
struct bio_job *job; struct bio_job *job;
unsigned long type = (unsigned long) arg; unsigned long type = (unsigned long) arg;
...@@ -196,20 +218,11 @@ void *bioProcessBackgroundJobs(void *arg) { ...@@ -196,20 +218,11 @@ void *bioProcessBackgroundJobs(void *arg) {
/* Process the job accordingly to its type. */ /* Process the job accordingly to its type. */
if (type == BIO_CLOSE_FILE) { if (type == BIO_CLOSE_FILE) {
close((long)job->arg1); close(job->fd);
} else if (type == BIO_AOF_FSYNC) { } else if (type == BIO_AOF_FSYNC) {
redis_fsync((long)job->arg1); redis_fsync(job->fd);
} else if (type == BIO_LAZY_FREE) { } else if (type == BIO_LAZY_FREE) {
/* What we free changes depending on what arguments are set: job->free_fn(job->free_args);
* arg1 -> free the object at pointer.
* arg2 & arg3 -> free two dictionaries (a Redis DB).
* only arg3 -> free the radix tree. */
if (job->arg1)
lazyfreeFreeObjectFromBioThread(job->arg1);
else if (job->arg2 && job->arg3)
lazyfreeFreeDatabaseFromBioThread(job->arg2,job->arg3);
else if (job->arg3)
lazyfreeFreeSlotsMapFromBioThread(job->arg3);
} else { } else {
serverPanic("Wrong job type in bioProcessBackgroundJobs()."); serverPanic("Wrong job type in bioProcessBackgroundJobs().");
} }
......
...@@ -30,13 +30,17 @@ ...@@ -30,13 +30,17 @@
#ifndef __BIO_H #ifndef __BIO_H
#define __BIO_H #define __BIO_H
typedef void lazy_free_fn(void *args[]);
/* Exported API */ /* Exported API */
void bioInit(void); void bioInit(void);
void bioCreateBackgroundJob(int type, void *arg1, void *arg2, void *arg3);
unsigned long long bioPendingJobsOfType(int type); unsigned long long bioPendingJobsOfType(int type);
unsigned long long bioWaitStepOfType(int type); unsigned long long bioWaitStepOfType(int type);
time_t bioOlderJobOfType(int type); time_t bioOlderJobOfType(int type);
void bioKillThreads(void); void bioKillThreads(void);
void bioCreateCloseJob(int fd);
void bioCreateFsyncJob(int fd);
void bioCreateLazyFreeJob(lazy_free_fn free_fn, int arg_count, ...);
/* Background job opcodes */ /* Background job opcodes */
#define BIO_CLOSE_FILE 0 /* Deferred close(2) syscall. */ #define BIO_CLOSE_FILE 0 /* Deferred close(2) syscall. */
......
...@@ -611,7 +611,7 @@ void bitopCommand(client *c) { ...@@ -611,7 +611,7 @@ void bitopCommand(client *c) {
else if((opname[0] == 'n' || opname[0] == 'N') && !strcasecmp(opname,"not")) else if((opname[0] == 'n' || opname[0] == 'N') && !strcasecmp(opname,"not"))
op = BITOP_NOT; op = BITOP_NOT;
else { else {
addReply(c,shared.syntaxerr); addReplyErrorObject(c,shared.syntaxerr);
return; return;
} }
...@@ -813,7 +813,7 @@ void bitcountCommand(client *c) { ...@@ -813,7 +813,7 @@ void bitcountCommand(client *c) {
end = strlen-1; end = strlen-1;
} else { } else {
/* Syntax error. */ /* Syntax error. */
addReply(c,shared.syntaxerr); addReplyErrorObject(c,shared.syntaxerr);
return; return;
} }
...@@ -878,7 +878,7 @@ void bitposCommand(client *c) { ...@@ -878,7 +878,7 @@ void bitposCommand(client *c) {
end = strlen-1; end = strlen-1;
} else { } else {
/* Syntax error. */ /* Syntax error. */
addReply(c,shared.syntaxerr); addReplyErrorObject(c,shared.syntaxerr);
return; return;
} }
...@@ -970,7 +970,7 @@ void bitfieldGeneric(client *c, int flags) { ...@@ -970,7 +970,7 @@ void bitfieldGeneric(client *c, int flags) {
} }
continue; continue;
} else { } else {
addReply(c,shared.syntaxerr); addReplyErrorObject(c,shared.syntaxerr);
zfree(ops); zfree(ops);
return; return;
} }
......
...@@ -89,6 +89,12 @@ void blockClient(client *c, int btype) { ...@@ -89,6 +89,12 @@ void blockClient(client *c, int btype) {
server.blocked_clients++; server.blocked_clients++;
server.blocked_clients_by_type[btype]++; server.blocked_clients_by_type[btype]++;
addClientToTimeoutTable(c); addClientToTimeoutTable(c);
if (btype == BLOCKED_PAUSE) {
listAddNodeTail(server.paused_clients, c);
c->paused_list_node = listLast(server.paused_clients);
/* Mark this client to execute its command */
c->flags |= CLIENT_PENDING_COMMAND;
}
} }
/* This function is called in the beforeSleep() function of the event loop /* This function is called in the beforeSleep() function of the event loop
...@@ -110,6 +116,11 @@ void processUnblockedClients(void) { ...@@ -110,6 +116,11 @@ void processUnblockedClients(void) {
* client is not blocked before to proceed, but things may change and * client is not blocked before to proceed, but things may change and
* the code is conceptually more correct this way. */ * the code is conceptually more correct this way. */
if (!(c->flags & CLIENT_BLOCKED)) { if (!(c->flags & CLIENT_BLOCKED)) {
/* If we have a queued command, execute it now. */
if (processPendingCommandsAndResetClient(c) == C_ERR) {
continue;
}
/* Then process client if it has more data in it's buffer. */
if (c->querybuf && sdslen(c->querybuf) > 0) { if (c->querybuf && sdslen(c->querybuf) > 0) {
processInputBuffer(c); processInputBuffer(c);
} }
...@@ -154,6 +165,9 @@ void unblockClient(client *c) { ...@@ -154,6 +165,9 @@ void unblockClient(client *c) {
} else if (c->btype == BLOCKED_MODULE) { } else if (c->btype == BLOCKED_MODULE) {
if (moduleClientIsBlockedOnKeys(c)) unblockClientWaitingData(c); if (moduleClientIsBlockedOnKeys(c)) unblockClientWaitingData(c);
unblockClientFromModule(c); unblockClientFromModule(c);
} else if (c->btype == BLOCKED_PAUSE) {
listDelNode(server.paused_clients,c->paused_list_node);
c->paused_list_node = NULL;
} else { } else {
serverPanic("Unknown btype in unblockClient()."); serverPanic("Unknown btype in unblockClient().");
} }
...@@ -200,9 +214,16 @@ void disconnectAllBlockedClients(void) { ...@@ -200,9 +214,16 @@ void disconnectAllBlockedClients(void) {
client *c = listNodeValue(ln); client *c = listNodeValue(ln);
if (c->flags & CLIENT_BLOCKED) { if (c->flags & CLIENT_BLOCKED) {
addReplySds(c,sdsnew( /* PAUSED clients are an exception, when they'll be unblocked, the
* command processing will start from scratch, and the command will
* be either executed or rejected. (unlike LIST blocked clients for
* which the command is already in progress in a way. */
if (c->btype == BLOCKED_PAUSE)
continue;
addReplyError(c,
"-UNBLOCKED force unblock from blocking operation, " "-UNBLOCKED force unblock from blocking operation, "
"instance state changed (master -> replica?)\r\n")); "instance state changed (master -> replica?)");
unblockClient(c); unblockClient(c);
c->flags |= CLIENT_CLOSE_AFTER_REPLY; c->flags |= CLIENT_CLOSE_AFTER_REPLY;
} }
......
...@@ -30,6 +30,12 @@ ...@@ -30,6 +30,12 @@
#include "server.h" #include "server.h"
#include <unistd.h> #include <unistd.h>
typedef struct {
int process_type; /* AOF or RDB child? */
int on_exit; /* COW size of active or exited child */
size_t cow_size; /* Copy on write size. */
} child_info_data;
/* Open a child-parent channel used in order to move information about the /* Open a child-parent channel used in order to move information about the
* RDB / AOF saving process from the child to the parent (for instance * RDB / AOF saving process from the child to the parent (for instance
* the amount of copy on write memory used) */ * the amount of copy on write memory used) */
...@@ -41,7 +47,7 @@ void openChildInfoPipe(void) { ...@@ -41,7 +47,7 @@ void openChildInfoPipe(void) {
} else if (anetNonBlock(NULL,server.child_info_pipe[0]) != ANET_OK) { } else if (anetNonBlock(NULL,server.child_info_pipe[0]) != ANET_OK) {
closeChildInfoPipe(); closeChildInfoPipe();
} else { } else {
memset(&server.child_info_data,0,sizeof(server.child_info_data)); server.child_info_nread = 0;
} }
} }
...@@ -54,34 +60,76 @@ void closeChildInfoPipe(void) { ...@@ -54,34 +60,76 @@ void closeChildInfoPipe(void) {
close(server.child_info_pipe[1]); close(server.child_info_pipe[1]);
server.child_info_pipe[0] = -1; server.child_info_pipe[0] = -1;
server.child_info_pipe[1] = -1; server.child_info_pipe[1] = -1;
server.child_info_nread = 0;
} }
} }
/* Send COW data to parent. The child should call this function after populating /* Send COW data to parent. */
* the corresponding fields it want to sent (according to the process type). */ void sendChildInfo(int process_type, int on_exit, size_t cow_size) {
void sendChildInfo(int ptype) {
if (server.child_info_pipe[1] == -1) return; if (server.child_info_pipe[1] == -1) return;
server.child_info_data.magic = CHILD_INFO_MAGIC;
server.child_info_data.process_type = ptype; child_info_data buffer = {.process_type = process_type, .on_exit = on_exit, .cow_size = cow_size};
ssize_t wlen = sizeof(server.child_info_data); ssize_t wlen = sizeof(buffer);
if (write(server.child_info_pipe[1],&server.child_info_data,wlen) != wlen) {
if (write(server.child_info_pipe[1],&buffer,wlen) != wlen) {
/* Nothing to do on error, this will be detected by the other side. */ /* Nothing to do on error, this will be detected by the other side. */
} }
} }
/* Receive COW data from parent. */ /* Update COW data. */
void updateChildInfo(int process_type, int on_exit, size_t cow_size) {
if (!on_exit) {
server.stat_current_cow_bytes = cow_size;
return;
}
if (process_type == CHILD_TYPE_RDB) {
server.stat_rdb_cow_bytes = cow_size;
} else if (process_type == CHILD_TYPE_AOF) {
server.stat_aof_cow_bytes = cow_size;
} else if (process_type == CHILD_TYPE_MODULE) {
server.stat_module_cow_bytes = cow_size;
}
}
/* Read COW info data from the pipe.
* if complete data read into the buffer, process type, copy-on-write type and copy-on-write size
* are stored into *process_type, *on_exit and *cow_size respectively and returns 1.
* otherwise, the partial data is left in the buffer, waiting for the next read, and returns 0. */
int readChildInfo(int *process_type, int *on_exit, size_t *cow_size) {
/* We are using here a static buffer in combination with the server.child_info_nread to handle short reads */
static child_info_data buffer;
ssize_t wlen = sizeof(buffer);
/* Do not overlap */
if (server.child_info_nread == wlen) server.child_info_nread = 0;
int nread = read(server.child_info_pipe[0], (char *)&buffer + server.child_info_nread, wlen - server.child_info_nread);
if (nread > 0) {
server.child_info_nread += nread;
}
/* We have complete child info */
if (server.child_info_nread == wlen) {
*process_type = buffer.process_type;
*on_exit = buffer.on_exit;
*cow_size = buffer.cow_size;
return 1;
} else {
return 0;
}
}
/* Receive COW data from child. */
void receiveChildInfo(void) { void receiveChildInfo(void) {
if (server.child_info_pipe[0] == -1) return; if (server.child_info_pipe[0] == -1) return;
ssize_t wlen = sizeof(server.child_info_data);
if (read(server.child_info_pipe[0],&server.child_info_data,wlen) == wlen && int process_type;
server.child_info_data.magic == CHILD_INFO_MAGIC) int on_exit;
{ size_t cow_size;
if (server.child_info_data.process_type == CHILD_TYPE_RDB) {
server.stat_rdb_cow_bytes = server.child_info_data.cow_size; /* Drain the pipe and update child info so that we get the final message. */
} else if (server.child_info_data.process_type == CHILD_TYPE_AOF) { while (readChildInfo(&process_type, &on_exit, &cow_size)) {
server.stat_aof_cow_bytes = server.child_info_data.cow_size; updateChildInfo(process_type, on_exit, cow_size);
} else if (server.child_info_data.process_type == CHILD_TYPE_MODULE) {
server.stat_module_cow_bytes = server.child_info_data.cow_size;
}
} }
} }
...@@ -47,7 +47,7 @@ ...@@ -47,7 +47,7 @@
clusterNode *myself = NULL; clusterNode *myself = NULL;
clusterNode *createClusterNode(char *nodename, int flags); clusterNode *createClusterNode(char *nodename, int flags);
int clusterAddNode(clusterNode *node); void clusterAddNode(clusterNode *node);
void clusterAcceptHandler(aeEventLoop *el, int fd, void *privdata, int mask); void clusterAcceptHandler(aeEventLoop *el, int fd, void *privdata, int mask);
void clusterReadHandler(connection *conn); void clusterReadHandler(connection *conn);
void clusterSendPing(clusterLink *link, int type); void clusterSendPing(clusterLink *link, int type);
...@@ -961,12 +961,12 @@ void freeClusterNode(clusterNode *n) { ...@@ -961,12 +961,12 @@ void freeClusterNode(clusterNode *n) {
} }
/* Add a node to the nodes hash table */ /* Add a node to the nodes hash table */
int clusterAddNode(clusterNode *node) { void clusterAddNode(clusterNode *node) {
int retval; int retval;
retval = dictAdd(server.cluster->nodes, retval = dictAdd(server.cluster->nodes,
sdsnewlen(node->name,CLUSTER_NAMELEN), node); sdsnewlen(node->name,CLUSTER_NAMELEN), node);
return (retval == DICT_OK) ? C_OK : C_ERR; serverAssert(retval == DICT_OK);
} }
/* Remove a node from the cluster. The function performs the high level /* Remove a node from the cluster. The function performs the high level
...@@ -2164,7 +2164,7 @@ int clusterProcessPacket(clusterLink *link) { ...@@ -2164,7 +2164,7 @@ int clusterProcessPacket(clusterLink *link) {
resetManualFailover(); resetManualFailover();
server.cluster->mf_end = now + CLUSTER_MF_TIMEOUT; server.cluster->mf_end = now + CLUSTER_MF_TIMEOUT;
server.cluster->mf_slave = sender; server.cluster->mf_slave = sender;
pauseClients(now+(CLUSTER_MF_TIMEOUT*CLUSTER_MF_PAUSE_MULT)); pauseClients(now+(CLUSTER_MF_TIMEOUT*CLUSTER_MF_PAUSE_MULT),CLIENT_PAUSE_WRITE);
serverLog(LL_WARNING,"Manual failover requested by replica %.40s.", serverLog(LL_WARNING,"Manual failover requested by replica %.40s.",
sender->name); sender->name);
/* We need to send a ping message to the replica, as it would carry /* We need to send a ping message to the replica, as it would carry
...@@ -3421,9 +3421,8 @@ void clusterHandleSlaveMigration(int max_slaves) { ...@@ -3421,9 +3421,8 @@ void clusterHandleSlaveMigration(int max_slaves) {
* The function can be used both to initialize the manual failover state at * The function can be used both to initialize the manual failover state at
* startup or to abort a manual failover in progress. */ * startup or to abort a manual failover in progress. */
void resetManualFailover(void) { void resetManualFailover(void) {
if (server.cluster->mf_end && clientsArePaused()) { if (server.cluster->mf_end) {
server.clients_pause_end_time = 0; checkClientPauseTimeoutAndReturnIfPaused();
clientsArePaused(); /* Just use the side effect of the function. */
} }
server.cluster->mf_end = 0; /* No manual failover in progress. */ server.cluster->mf_end = 0; /* No manual failover in progress. */
server.cluster->mf_can_start = 0; server.cluster->mf_can_start = 0;
...@@ -4357,28 +4356,49 @@ void clusterCommand(client *c) { ...@@ -4357,28 +4356,49 @@ void clusterCommand(client *c) {
if (c->argc == 2 && !strcasecmp(c->argv[1]->ptr,"help")) { if (c->argc == 2 && !strcasecmp(c->argv[1]->ptr,"help")) {
const char *help[] = { const char *help[] = {
"ADDSLOTS <slot> [slot ...] -- Assign slots to current node.", "ADDSLOTS <slot> [<slot> ...]",
"BUMPEPOCH -- Advance the cluster config epoch.", " Assign slots to current node.",
"COUNT-failure-reports <node-id> -- Return number of failure reports for <node-id>.", "BUMPEPOCH",
"COUNTKEYSINSLOT <slot> - Return the number of keys in <slot>.", " Advance the cluster config epoch.",
"DELSLOTS <slot> [slot ...] -- Delete slots information from current node.", "COUNT-FAILURE-REPORTS <node-id>",
"FAILOVER [force|takeover] -- Promote current replica node to being a master.", " Return number of failure reports for <node-id>.",
"FORGET <node-id> -- Remove a node from the cluster.", "COUNTKEYSINSLOT <slot>",
"GETKEYSINSLOT <slot> <count> -- Return key names stored by current node in a slot.", " Return the number of keys in <slot>.",
"FLUSHSLOTS -- Delete current node own slots information.", "DELSLOTS <slot> [<slot> ...]",
"INFO - Return information about the cluster.", " Delete slots information from current node.",
"KEYSLOT <key> -- Return the hash slot for <key>.", "FAILOVER [FORCE|TAKEOVER]",
"MEET <ip> <port> [bus-port] -- Connect nodes into a working cluster.", " Promote current replica node to being a master.",
"MYID -- Return the node id.", "FORGET <node-id>",
"NODES -- Return cluster configuration seen by node. Output format:", " Remove a node from the cluster.",
" <id> <ip:port> <flags> <master> <pings> <pongs> <epoch> <link> <slot> ... <slot>", "GETKEYSINSLOT <slot> <count>",
"REPLICATE <node-id> -- Configure current node as replica to <node-id>.", " Return key names stored by current node in a slot.",
"RESET [hard|soft] -- Reset current node (default: soft).", "FLUSHSLOTS",
"SET-config-epoch <epoch> - Set config epoch of current node.", " Delete current node own slots information.",
"SETSLOT <slot> (importing|migrating|stable|node <node-id>) -- Set slot state.", "INFO",
"REPLICAS <node-id> -- Return <node-id> replicas.", " Return information about the cluster.",
"SAVECONFIG - Force saving cluster configuration on disk.", "KEYSLOT <key>",
"SLOTS -- Return information about slots range mappings. Each range is made of:", " Return the hash slot for <key>.",
"MEET <ip> <port> [<bus-port>]",
" Connect nodes into a working cluster.",
"MYID",
" Return the node id.",
"NODES",
" Return cluster configuration seen by node. Output format:",
" <id> <ip:port> <flags> <master> <pings> <pongs> <epoch> <link> <slot> ...",
"REPLICATE <node-id>",
" Configure current node as replica to <node-id>.",
"RESET [HARD|SOFT]",
" Reset current node (default: soft).",
"SET-CONFIG-EPOCH <epoch>",
" Set config epoch of current node.",
"SETSLOT <slot> (IMPORTING|MIGRATING|STABLE|NODE <node-id>)",
" Set slot state.",
"REPLICAS <node-id>",
" Return <node-id> replicas.",
"SAVECONFIG",
" Force saving cluster configuration on disk.",
"SLOTS",
" Return information about slots range mappings. Each range is made of:",
" start, end, master and replicas IP addresses, ports and ids", " start, end, master and replicas IP addresses, ports and ids",
NULL NULL
}; };
...@@ -4820,7 +4840,7 @@ NULL ...@@ -4820,7 +4840,7 @@ NULL
takeover = 1; takeover = 1;
force = 1; /* Takeover also implies force. */ force = 1; /* Takeover also implies force. */
} else { } else {
addReply(c,shared.syntaxerr); addReplyErrorObject(c,shared.syntaxerr);
return; return;
} }
} }
...@@ -4911,7 +4931,7 @@ NULL ...@@ -4911,7 +4931,7 @@ NULL
} else if (!strcasecmp(c->argv[2]->ptr,"soft")) { } else if (!strcasecmp(c->argv[2]->ptr,"soft")) {
hard = 0; hard = 0;
} else { } else {
addReply(c,shared.syntaxerr); addReplyErrorObject(c,shared.syntaxerr);
return; return;
} }
} }
...@@ -5049,7 +5069,7 @@ void restoreCommand(client *c) { ...@@ -5049,7 +5069,7 @@ void restoreCommand(client *c) {
} }
j++; /* Consume additional arg. */ j++; /* Consume additional arg. */
} else { } else {
addReply(c,shared.syntaxerr); addReplyErrorObject(c,shared.syntaxerr);
return; return;
} }
} }
...@@ -5057,7 +5077,7 @@ void restoreCommand(client *c) { ...@@ -5057,7 +5077,7 @@ void restoreCommand(client *c) {
/* Make sure this key does not already exist here... */ /* Make sure this key does not already exist here... */
robj *key = c->argv[1]; robj *key = c->argv[1];
if (!replace && lookupKeyWrite(c->db,key) != NULL) { if (!replace && lookupKeyWrite(c->db,key) != NULL) {
addReply(c,shared.busykeyerr); addReplyErrorObject(c,shared.busykeyerr);
return; return;
} }
...@@ -5170,8 +5190,7 @@ migrateCachedSocket* migrateGetSocket(client *c, robj *host, robj *port, long ti ...@@ -5170,8 +5190,7 @@ migrateCachedSocket* migrateGetSocket(client *c, robj *host, robj *port, long ti
conn = server.tls_cluster ? connCreateTLS() : connCreateSocket(); conn = server.tls_cluster ? connCreateTLS() : connCreateSocket();
if (connBlockingConnect(conn, c->argv[1]->ptr, atoi(c->argv[2]->ptr), timeout) if (connBlockingConnect(conn, c->argv[1]->ptr, atoi(c->argv[2]->ptr), timeout)
!= C_OK) { != C_OK) {
addReplySds(c, addReplyError(c,"-IOERR error or timeout connecting to the client");
sdsnew("-IOERR error or timeout connecting to the client\r\n"));
connClose(conn); connClose(conn);
sdsfree(name); sdsfree(name);
return NULL; return NULL;
...@@ -5259,14 +5278,14 @@ void migrateCommand(client *c) { ...@@ -5259,14 +5278,14 @@ void migrateCommand(client *c) {
replace = 1; replace = 1;
} else if (!strcasecmp(c->argv[j]->ptr,"auth")) { } else if (!strcasecmp(c->argv[j]->ptr,"auth")) {
if (!moreargs) { if (!moreargs) {
addReply(c,shared.syntaxerr); addReplyErrorObject(c,shared.syntaxerr);
return; return;
} }
j++; j++;
password = c->argv[j]->ptr; password = c->argv[j]->ptr;
} else if (!strcasecmp(c->argv[j]->ptr,"auth2")) { } else if (!strcasecmp(c->argv[j]->ptr,"auth2")) {
if (moreargs < 2) { if (moreargs < 2) {
addReply(c,shared.syntaxerr); addReplyErrorObject(c,shared.syntaxerr);
return; return;
} }
username = c->argv[++j]->ptr; username = c->argv[++j]->ptr;
...@@ -5282,7 +5301,7 @@ void migrateCommand(client *c) { ...@@ -5282,7 +5301,7 @@ void migrateCommand(client *c) {
num_keys = c->argc - j - 1; num_keys = c->argc - j - 1;
break; /* All the remaining args are keys. */ break; /* All the remaining args are keys. */
} else { } else {
addReply(c,shared.syntaxerr); addReplyErrorObject(c,shared.syntaxerr);
return; return;
} }
} }
...@@ -5763,7 +5782,7 @@ clusterNode *getNodeByQuery(client *c, struct redisCommand *cmd, robj **argv, in ...@@ -5763,7 +5782,7 @@ clusterNode *getNodeByQuery(client *c, struct redisCommand *cmd, robj **argv, in
* cluster is down. */ * cluster is down. */
if (error_code) *error_code = CLUSTER_REDIR_DOWN_STATE; if (error_code) *error_code = CLUSTER_REDIR_DOWN_STATE;
return NULL; return NULL;
} else if (!(cmd->flags & CMD_READONLY) && !(cmd->proc == evalCommand) } else if ((cmd->flags & CMD_WRITE) && !(cmd->proc == evalCommand)
&& !(cmd->proc == evalShaCommand)) && !(cmd->proc == evalShaCommand))
{ {
/* The cluster is configured to allow read only commands /* The cluster is configured to allow read only commands
...@@ -5812,11 +5831,10 @@ clusterNode *getNodeByQuery(client *c, struct redisCommand *cmd, robj **argv, in ...@@ -5812,11 +5831,10 @@ clusterNode *getNodeByQuery(client *c, struct redisCommand *cmd, robj **argv, in
/* Handle the read-only client case reading from a slave: if this /* Handle the read-only client case reading from a slave: if this
* node is a slave and the request is about a hash slot our master * node is a slave and the request is about a hash slot our master
* is serving, we can reply without redirection. */ * is serving, we can reply without redirection. */
int is_readonly_command = (c->cmd->flags & CMD_READONLY) || int is_write_command = (c->cmd->flags & CMD_WRITE) ||
(c->cmd->proc == execCommand && !(c->mstate.cmd_inv_flags & CMD_READONLY)); (c->cmd->proc == execCommand && (c->mstate.cmd_flags & CMD_WRITE));
if (c->flags & CLIENT_READONLY && if (c->flags & CLIENT_READONLY &&
(is_readonly_command || cmd->proc == evalCommand || (!is_write_command || cmd->proc == evalCommand || cmd->proc == evalShaCommand) &&
cmd->proc == evalShaCommand) &&
nodeIsSlave(myself) && nodeIsSlave(myself) &&
myself->slaveof == n) myself->slaveof == n)
{ {
...@@ -5838,23 +5856,23 @@ clusterNode *getNodeByQuery(client *c, struct redisCommand *cmd, robj **argv, in ...@@ -5838,23 +5856,23 @@ clusterNode *getNodeByQuery(client *c, struct redisCommand *cmd, robj **argv, in
* be set to the hash slot that caused the redirection. */ * be set to the hash slot that caused the redirection. */
void clusterRedirectClient(client *c, clusterNode *n, int hashslot, int error_code) { void clusterRedirectClient(client *c, clusterNode *n, int hashslot, int error_code) {
if (error_code == CLUSTER_REDIR_CROSS_SLOT) { if (error_code == CLUSTER_REDIR_CROSS_SLOT) {
addReplySds(c,sdsnew("-CROSSSLOT Keys in request don't hash to the same slot\r\n")); addReplyError(c,"-CROSSSLOT Keys in request don't hash to the same slot");
} else if (error_code == CLUSTER_REDIR_UNSTABLE) { } else if (error_code == CLUSTER_REDIR_UNSTABLE) {
/* The request spawns multiple keys in the same slot, /* The request spawns multiple keys in the same slot,
* but the slot is not "stable" currently as there is * but the slot is not "stable" currently as there is
* a migration or import in progress. */ * a migration or import in progress. */
addReplySds(c,sdsnew("-TRYAGAIN Multiple keys request during rehashing of slot\r\n")); addReplyError(c,"-TRYAGAIN Multiple keys request during rehashing of slot");
} else if (error_code == CLUSTER_REDIR_DOWN_STATE) { } else if (error_code == CLUSTER_REDIR_DOWN_STATE) {
addReplySds(c,sdsnew("-CLUSTERDOWN The cluster is down\r\n")); addReplyError(c,"-CLUSTERDOWN The cluster is down");
} else if (error_code == CLUSTER_REDIR_DOWN_RO_STATE) { } else if (error_code == CLUSTER_REDIR_DOWN_RO_STATE) {
addReplySds(c,sdsnew("-CLUSTERDOWN The cluster is down and only accepts read commands\r\n")); addReplyError(c,"-CLUSTERDOWN The cluster is down and only accepts read commands");
} else if (error_code == CLUSTER_REDIR_DOWN_UNBOUND) { } else if (error_code == CLUSTER_REDIR_DOWN_UNBOUND) {
addReplySds(c,sdsnew("-CLUSTERDOWN Hash slot not served\r\n")); addReplyError(c,"-CLUSTERDOWN Hash slot not served");
} else if (error_code == CLUSTER_REDIR_MOVED || } else if (error_code == CLUSTER_REDIR_MOVED ||
error_code == CLUSTER_REDIR_ASK) error_code == CLUSTER_REDIR_ASK)
{ {
addReplySds(c,sdscatprintf(sdsempty(), addReplyErrorSds(c,sdscatprintf(sdsempty(),
"-%s %d %s:%d\r\n", "-%s %d %s:%d",
(error_code == CLUSTER_REDIR_ASK) ? "ASK" : "MOVED", (error_code == CLUSTER_REDIR_ASK) ? "ASK" : "MOVED",
hashslot,n->ip,n->port)); hashslot,n->ip,n->port));
} else { } else {
...@@ -5901,7 +5919,7 @@ int clusterRedirectBlockedClientIfNeeded(client *c) { ...@@ -5901,7 +5919,7 @@ int clusterRedirectBlockedClientIfNeeded(client *c) {
/* if the client is read-only and attempting to access key that our /* if the client is read-only and attempting to access key that our
* replica can handle, allow it. */ * replica can handle, allow it. */
if ((c->flags & CLIENT_READONLY) && if ((c->flags & CLIENT_READONLY) &&
(c->lastcmd->flags & CMD_READONLY) && !(c->lastcmd->flags & CMD_WRITE) &&
nodeIsSlave(myself) && myself->slaveof == node) nodeIsSlave(myself) && myself->slaveof == node)
{ {
node = myself; node = myself;
......
...@@ -40,7 +40,7 @@ typedef struct clusterLink { ...@@ -40,7 +40,7 @@ typedef struct clusterLink {
sds sndbuf; /* Packet send buffer */ sds sndbuf; /* Packet send buffer */
char *rcvbuf; /* Packet reception buffer */ char *rcvbuf; /* Packet reception buffer */
size_t rcvbuf_len; /* Used size of rcvbuf */ size_t rcvbuf_len; /* Used size of rcvbuf */
size_t rcvbuf_alloc; /* Used size of rcvbuf */ size_t rcvbuf_alloc; /* Allocated size of rcvbuf */
struct clusterNode *node; /* Node related to this link if any, or NULL */ struct clusterNode *node; /* Node related to this link if any, or NULL */
} clusterLink; } clusterLink;
......
...@@ -166,6 +166,15 @@ typedef struct stringConfigData { ...@@ -166,6 +166,15 @@ typedef struct stringConfigData {
be stored as a NULL value. */ be stored as a NULL value. */
} stringConfigData; } stringConfigData;
typedef struct sdsConfigData {
sds *config; /* Pointer to the server config this value is stored in. */
const char *default_value; /* Default value of the config on rewrite. */
int (*is_valid_fn)(sds val, char **err); /* Optional function to check validity of new value (generic doc above) */
int (*update_fn)(sds val, sds prev, char **err); /* Optional function to apply new value at runtime (generic doc above) */
int convert_empty_to_null; /* Boolean indicating if empty SDS strings should
be stored as a NULL value. */
} sdsConfigData;
typedef struct enumConfigData { typedef struct enumConfigData {
int *config; /* The pointer to the server config this value is stored in */ int *config; /* The pointer to the server config this value is stored in */
configEnum *enum_value; /* The underlying enum type this data represents */ configEnum *enum_value; /* The underlying enum type this data represents */
...@@ -212,6 +221,7 @@ typedef struct numericConfigData { ...@@ -212,6 +221,7 @@ typedef struct numericConfigData {
typedef union typeData { typedef union typeData {
boolConfigData yesno; boolConfigData yesno;
stringConfigData string; stringConfigData string;
sdsConfigData sds;
enumConfigData enumd; enumConfigData enumd;
numericConfigData numeric; numericConfigData numeric;
} typeData; } typeData;
...@@ -512,7 +522,7 @@ void loadServerConfigFromString(char *config) { ...@@ -512,7 +522,7 @@ void loadServerConfigFromString(char *config) {
} }
server.repl_state = REPL_STATE_CONNECT; server.repl_state = REPL_STATE_CONNECT;
} else if (!strcasecmp(argv[0],"requirepass") && argc == 2) { } else if (!strcasecmp(argv[0],"requirepass") && argc == 2) {
if (strlen(argv[1]) > CONFIG_AUTHPASS_MAX_LEN) { if (sdslen(argv[1]) > CONFIG_AUTHPASS_MAX_LEN) {
err = "Password is longer than CONFIG_AUTHPASS_MAX_LEN"; err = "Password is longer than CONFIG_AUTHPASS_MAX_LEN";
goto loaderr; goto loaderr;
} }
...@@ -524,10 +534,10 @@ void loadServerConfigFromString(char *config) { ...@@ -524,10 +534,10 @@ void loadServerConfigFromString(char *config) {
sdsfree(server.requirepass); sdsfree(server.requirepass);
server.requirepass = NULL; server.requirepass = NULL;
if (sdslen(argv[1])) { if (sdslen(argv[1])) {
sds aclop = sdscatprintf(sdsempty(),">%s",argv[1]); sds aclop = sdscatlen(sdsnew(">"), argv[1], sdslen(argv[1]));
ACLSetUser(DefaultUser,aclop,sdslen(aclop)); ACLSetUser(DefaultUser,aclop,sdslen(aclop));
sdsfree(aclop); sdsfree(aclop);
server.requirepass = sdsnew(argv[1]); server.requirepass = sdsdup(argv[1]);
} else { } else {
ACLSetUser(DefaultUser,"nopass",-1); ACLSetUser(DefaultUser,"nopass",-1);
} }
...@@ -751,10 +761,10 @@ void configSetCommand(client *c) { ...@@ -751,10 +761,10 @@ void configSetCommand(client *c) {
sdsfree(server.requirepass); sdsfree(server.requirepass);
server.requirepass = NULL; server.requirepass = NULL;
if (sdslen(o->ptr)) { if (sdslen(o->ptr)) {
sds aclop = sdscatprintf(sdsempty(),">%s",(char*)o->ptr); sds aclop = sdscatlen(sdsnew(">"), o->ptr, sdslen(o->ptr));
ACLSetUser(DefaultUser,aclop,sdslen(aclop)); ACLSetUser(DefaultUser,aclop,sdslen(aclop));
sdsfree(aclop); sdsfree(aclop);
server.requirepass = sdsnew(o->ptr); server.requirepass = sdsdup(o->ptr);
} else { } else {
ACLSetUser(DefaultUser,"nopass",-1); ACLSetUser(DefaultUser,"nopass",-1);
} }
...@@ -905,7 +915,7 @@ badfmt: /* Bad format errors */ ...@@ -905,7 +915,7 @@ badfmt: /* Bad format errors */
addReplyBulkCString(c,_var ? _var : ""); \ addReplyBulkCString(c,_var ? _var : ""); \
matches++; \ matches++; \
} \ } \
} while(0); } while(0)
#define config_get_bool_field(_name,_var) do { \ #define config_get_bool_field(_name,_var) do { \
if (stringmatch(pattern,_name,1)) { \ if (stringmatch(pattern,_name,1)) { \
...@@ -913,7 +923,7 @@ badfmt: /* Bad format errors */ ...@@ -913,7 +923,7 @@ badfmt: /* Bad format errors */
addReplyBulkCString(c,_var ? "yes" : "no"); \ addReplyBulkCString(c,_var ? "yes" : "no"); \
matches++; \ matches++; \
} \ } \
} while(0); } while(0)
#define config_get_numerical_field(_name,_var) do { \ #define config_get_numerical_field(_name,_var) do { \
if (stringmatch(pattern,_name,1)) { \ if (stringmatch(pattern,_name,1)) { \
...@@ -922,8 +932,7 @@ badfmt: /* Bad format errors */ ...@@ -922,8 +932,7 @@ badfmt: /* Bad format errors */
addReplyBulkCString(c,buf); \ addReplyBulkCString(c,buf); \
matches++; \ matches++; \
} \ } \
} while(0); } while(0)
void configGetCommand(client *c) { void configGetCommand(client *c) {
robj *o = c->argv[2]; robj *o = c->argv[2];
...@@ -1330,6 +1339,28 @@ void rewriteConfigStringOption(struct rewriteConfigState *state, const char *opt ...@@ -1330,6 +1339,28 @@ void rewriteConfigStringOption(struct rewriteConfigState *state, const char *opt
rewriteConfigRewriteLine(state,option,line,force); rewriteConfigRewriteLine(state,option,line,force);
} }
/* Rewrite a SDS string option. */
void rewriteConfigSdsOption(struct rewriteConfigState *state, const char *option, sds value, const sds defvalue) {
int force = 1;
sds line;
/* If there is no value set, we don't want the SDS option
* to be present in the configuration at all. */
if (value == NULL) {
rewriteConfigMarkAsProcessed(state, option);
return;
}
/* Set force to zero if the value is set to its default. */
if (defvalue && sdscmp(value, defvalue) == 0) force = 0;
line = sdsnew(option);
line = sdscatlen(line, " ", 1);
line = sdscatrepr(line, value, sdslen(value));
rewriteConfigRewriteLine(state, option, line, force);
}
/* Rewrite a numerical (long long range) option. */ /* Rewrite a numerical (long long range) option. */
void rewriteConfigNumericalOption(struct rewriteConfigState *state, const char *option, long long value, long long defvalue) { void rewriteConfigNumericalOption(struct rewriteConfigState *state, const char *option, long long value, long long defvalue) {
int force = value != defvalue; int force = value != defvalue;
...@@ -1802,22 +1833,14 @@ static void boolConfigRewrite(typeData data, const char *name, struct rewriteCon ...@@ -1802,22 +1833,14 @@ static void boolConfigRewrite(typeData data, const char *name, struct rewriteCon
/* String Configs */ /* String Configs */
static void stringConfigInit(typeData data) { static void stringConfigInit(typeData data) {
if (data.string.convert_empty_to_null) { *data.string.config = (data.string.convert_empty_to_null && !data.string.default_value) ? NULL : zstrdup(data.string.default_value);
*data.string.config = data.string.default_value ? zstrdup(data.string.default_value) : NULL;
} else {
*data.string.config = zstrdup(data.string.default_value);
}
} }
static int stringConfigSet(typeData data, sds value, int update, char **err) { static int stringConfigSet(typeData data, sds value, int update, char **err) {
if (data.string.is_valid_fn && !data.string.is_valid_fn(value, err)) if (data.string.is_valid_fn && !data.string.is_valid_fn(value, err))
return 0; return 0;
char *prev = *data.string.config; char *prev = *data.string.config;
if (data.string.convert_empty_to_null) { *data.string.config = (data.string.convert_empty_to_null && !value[0]) ? NULL : zstrdup(value);
*data.string.config = value[0] ? zstrdup(value) : NULL;
} else {
*data.string.config = zstrdup(value);
}
if (update && data.string.update_fn && !data.string.update_fn(*data.string.config, prev, err)) { if (update && data.string.update_fn && !data.string.update_fn(*data.string.config, prev, err)) {
zfree(*data.string.config); zfree(*data.string.config);
*data.string.config = prev; *data.string.config = prev;
...@@ -1835,6 +1858,38 @@ static void stringConfigRewrite(typeData data, const char *name, struct rewriteC ...@@ -1835,6 +1858,38 @@ static void stringConfigRewrite(typeData data, const char *name, struct rewriteC
rewriteConfigStringOption(state, name,*(data.string.config), data.string.default_value); rewriteConfigStringOption(state, name,*(data.string.config), data.string.default_value);
} }
/* SDS Configs */
static void sdsConfigInit(typeData data) {
*data.sds.config = (data.sds.convert_empty_to_null && !data.sds.default_value) ? NULL: sdsnew(data.sds.default_value);
}
static int sdsConfigSet(typeData data, sds value, int update, char **err) {
if (data.sds.is_valid_fn && !data.sds.is_valid_fn(value, err))
return 0;
sds prev = *data.sds.config;
*data.sds.config = (data.sds.convert_empty_to_null && (sdslen(value) == 0)) ? NULL : sdsdup(value);
if (update && data.sds.update_fn && !data.sds.update_fn(*data.sds.config, prev, err)) {
sdsfree(*data.sds.config);
*data.sds.config = prev;
return 0;
}
sdsfree(prev);
return 1;
}
static void sdsConfigGet(client *c, typeData data) {
if (*data.sds.config) {
addReplyBulkSds(c, sdsdup(*data.sds.config));
} else {
addReplyBulkCString(c, "");
}
}
static void sdsConfigRewrite(typeData data, const char *name, struct rewriteConfigState *state) {
rewriteConfigSdsOption(state, name, *(data.sds.config), data.sds.default_value ? sdsnew(data.sds.default_value) : NULL);
}
#define ALLOW_EMPTY_STRING 0 #define ALLOW_EMPTY_STRING 0
#define EMPTY_STRING_IS_NULL 1 #define EMPTY_STRING_IS_NULL 1
...@@ -1850,6 +1905,18 @@ static void stringConfigRewrite(typeData data, const char *name, struct rewriteC ...@@ -1850,6 +1905,18 @@ static void stringConfigRewrite(typeData data, const char *name, struct rewriteC
} \ } \
} }
#define createSDSConfig(name, alias, modifiable, empty_to_null, config_addr, default, is_valid, update) { \
embedCommonConfig(name, alias, modifiable) \
embedConfigInterface(sdsConfigInit, sdsConfigSet, sdsConfigGet, sdsConfigRewrite) \
.data.sds = { \
.config = &(config_addr), \
.default_value = (default), \
.is_valid_fn = (is_valid), \
.update_fn = (update), \
.convert_empty_to_null = (empty_to_null), \
} \
}
/* Enum configs */ /* Enum configs */
static void enumConfigInit(typeData data) { static void enumConfigInit(typeData data) {
*data.enumd.config = data.enumd.default_value; *data.enumd.config = data.enumd.default_value;
...@@ -2349,7 +2416,6 @@ standardConfig configs[] = { ...@@ -2349,7 +2416,6 @@ standardConfig configs[] = {
createStringConfig("pidfile", NULL, IMMUTABLE_CONFIG, EMPTY_STRING_IS_NULL, server.pidfile, NULL, NULL, NULL), createStringConfig("pidfile", NULL, IMMUTABLE_CONFIG, EMPTY_STRING_IS_NULL, server.pidfile, NULL, NULL, NULL),
createStringConfig("replica-announce-ip", "slave-announce-ip", MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.slave_announce_ip, NULL, NULL, NULL), createStringConfig("replica-announce-ip", "slave-announce-ip", MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.slave_announce_ip, NULL, NULL, NULL),
createStringConfig("masteruser", NULL, MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.masteruser, NULL, NULL, NULL), createStringConfig("masteruser", NULL, MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.masteruser, NULL, NULL, NULL),
createStringConfig("masterauth", NULL, MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.masterauth, NULL, NULL, NULL),
createStringConfig("cluster-announce-ip", NULL, MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.cluster_announce_ip, NULL, NULL, NULL), createStringConfig("cluster-announce-ip", NULL, MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.cluster_announce_ip, NULL, NULL, NULL),
createStringConfig("syslog-ident", NULL, IMMUTABLE_CONFIG, ALLOW_EMPTY_STRING, server.syslog_ident, "redis", NULL, NULL), createStringConfig("syslog-ident", NULL, IMMUTABLE_CONFIG, ALLOW_EMPTY_STRING, server.syslog_ident, "redis", NULL, NULL),
createStringConfig("dbfilename", NULL, MODIFIABLE_CONFIG, ALLOW_EMPTY_STRING, server.rdb_filename, "dump.rdb", isValidDBfilename, NULL), createStringConfig("dbfilename", NULL, MODIFIABLE_CONFIG, ALLOW_EMPTY_STRING, server.rdb_filename, "dump.rdb", isValidDBfilename, NULL),
...@@ -2358,6 +2424,10 @@ standardConfig configs[] = { ...@@ -2358,6 +2424,10 @@ standardConfig configs[] = {
createStringConfig("bio_cpulist", NULL, IMMUTABLE_CONFIG, EMPTY_STRING_IS_NULL, server.bio_cpulist, NULL, NULL, NULL), createStringConfig("bio_cpulist", NULL, IMMUTABLE_CONFIG, EMPTY_STRING_IS_NULL, server.bio_cpulist, NULL, NULL, NULL),
createStringConfig("aof_rewrite_cpulist", NULL, IMMUTABLE_CONFIG, EMPTY_STRING_IS_NULL, server.aof_rewrite_cpulist, NULL, NULL, NULL), createStringConfig("aof_rewrite_cpulist", NULL, IMMUTABLE_CONFIG, EMPTY_STRING_IS_NULL, server.aof_rewrite_cpulist, NULL, NULL, NULL),
createStringConfig("bgsave_cpulist", NULL, IMMUTABLE_CONFIG, EMPTY_STRING_IS_NULL, server.bgsave_cpulist, NULL, NULL, NULL), createStringConfig("bgsave_cpulist", NULL, IMMUTABLE_CONFIG, EMPTY_STRING_IS_NULL, server.bgsave_cpulist, NULL, NULL, NULL),
createStringConfig("ignore-warnings", NULL, MODIFIABLE_CONFIG, ALLOW_EMPTY_STRING, server.ignore_warnings, "", NULL, NULL),
/* SDS Configs */
createSDSConfig("masterauth", NULL, MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.masterauth, NULL, NULL, NULL),
/* Enum Configs */ /* Enum Configs */
createEnumConfig("supervised", NULL, IMMUTABLE_CONFIG, supervised_mode_enum, server.supervised_mode, SUPERVISED_NONE, NULL, NULL), createEnumConfig("supervised", NULL, IMMUTABLE_CONFIG, supervised_mode_enum, server.supervised_mode, SUPERVISED_NONE, NULL, NULL),
...@@ -2477,12 +2547,17 @@ void configCommand(client *c) { ...@@ -2477,12 +2547,17 @@ void configCommand(client *c) {
if (c->argc == 2 && !strcasecmp(c->argv[1]->ptr,"help")) { if (c->argc == 2 && !strcasecmp(c->argv[1]->ptr,"help")) {
const char *help[] = { const char *help[] = {
"GET <pattern> -- Return parameters matching the glob-like <pattern> and their values.", "GET <pattern>",
"SET <parameter> <value> -- Set parameter to value.", " Return parameters matching the glob-like <pattern> and their values.",
"RESETSTAT -- Reset statistics reported by INFO.", "SET <directive> <value>",
"REWRITE -- Rewrite the configuration file.", " Set the configuration <directive> to <value>.",
"RESETSTAT",
" Reset statistics reported by the INFO command.",
"REWRITE",
" Rewrite the configuration file.",
NULL NULL
}; };
addReplyHelp(c, help); addReplyHelp(c, help);
} else if (!strcasecmp(c->argv[1]->ptr,"set") && c->argc == 4) { } else if (!strcasecmp(c->argv[1]->ptr,"set") && c->argc == 4) {
configSetCommand(c); configSetCommand(c);
...@@ -2491,6 +2566,7 @@ NULL ...@@ -2491,6 +2566,7 @@ NULL
} else if (!strcasecmp(c->argv[1]->ptr,"resetstat") && c->argc == 2) { } else if (!strcasecmp(c->argv[1]->ptr,"resetstat") && c->argc == 2) {
resetServerStats(); resetServerStats();
resetCommandTableStats(); resetCommandTableStats();
resetErrorTableStats();
addReply(c,shared.ok); addReply(c,shared.ok);
} else if (!strcasecmp(c->argv[1]->ptr,"rewrite") && c->argc == 2) { } else if (!strcasecmp(c->argv[1]->ptr,"rewrite") && c->argc == 2) {
if (server.configfile == NULL) { if (server.configfile == NULL) {
......
...@@ -134,7 +134,7 @@ int crc64Test(int argc, char *argv[]) { ...@@ -134,7 +134,7 @@ int crc64Test(int argc, char *argv[]) {
printf("[calcula]: e9c6d914c4b8d9ca == %016" PRIx64 "\n", printf("[calcula]: e9c6d914c4b8d9ca == %016" PRIx64 "\n",
(uint64_t)_crc64(0, "123456789", 9)); (uint64_t)_crc64(0, "123456789", 9));
printf("[64speed]: e9c6d914c4b8d9ca == %016" PRIx64 "\n", printf("[64speed]: e9c6d914c4b8d9ca == %016" PRIx64 "\n",
(uint64_t)crc64(0, "123456789", 9)); (uint64_t)crc64(0, (unsigned char*)"123456789", 9));
char li[] = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed " char li[] = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed "
"do eiusmod tempor incididunt ut labore et dolore magna " "do eiusmod tempor incididunt ut labore et dolore magna "
"aliqua. Ut enim ad minim veniam, quis nostrud exercitation " "aliqua. Ut enim ad minim veniam, quis nostrud exercitation "
...@@ -146,7 +146,7 @@ int crc64Test(int argc, char *argv[]) { ...@@ -146,7 +146,7 @@ int crc64Test(int argc, char *argv[]) {
printf("[calcula]: c7794709e69683b3 == %016" PRIx64 "\n", printf("[calcula]: c7794709e69683b3 == %016" PRIx64 "\n",
(uint64_t)_crc64(0, li, sizeof(li))); (uint64_t)_crc64(0, li, sizeof(li)));
printf("[64speed]: c7794709e69683b3 == %016" PRIx64 "\n", printf("[64speed]: c7794709e69683b3 == %016" PRIx64 "\n",
(uint64_t)crc64(0, li, sizeof(li))); (uint64_t)crc64(0, (unsigned char*)li, sizeof(li)));
return 0; return 0;
} }
......
...@@ -35,7 +35,8 @@ void crcspeed64little_init(crcfn64 crcfn, uint64_t table[8][256]) { ...@@ -35,7 +35,8 @@ void crcspeed64little_init(crcfn64 crcfn, uint64_t table[8][256]) {
/* generate CRCs for all single byte sequences */ /* generate CRCs for all single byte sequences */
for (int n = 0; n < 256; n++) { for (int n = 0; n < 256; n++) {
table[0][n] = crcfn(0, &n, 1); unsigned char v = n;
table[0][n] = crcfn(0, &v, 1);
} }
/* generate nested CRC table for future slice-by-8 lookup */ /* generate nested CRC table for future slice-by-8 lookup */
......
...@@ -433,7 +433,7 @@ long long emptyDb(int dbnum, int flags, void(callback)(void*)) { ...@@ -433,7 +433,7 @@ long long emptyDb(int dbnum, int flags, void(callback)(void*)) {
/* Make sure the WATCHed keys are affected by the FLUSH* commands. /* Make sure the WATCHed keys are affected by the FLUSH* commands.
* Note that we need to call the function while the keys are still * Note that we need to call the function while the keys are still
* there. */ * there. */
signalFlushedDb(dbnum); signalFlushedDb(dbnum, async);
/* Empty redis database structure. */ /* Empty redis database structure. */
removed = emptyDbStructure(server.db, dbnum, async, callback); removed = emptyDbStructure(server.db, dbnum, async, callback);
...@@ -572,9 +572,20 @@ void signalModifiedKey(client *c, redisDb *db, robj *key) { ...@@ -572,9 +572,20 @@ void signalModifiedKey(client *c, redisDb *db, robj *key) {
trackingInvalidateKey(c,key); trackingInvalidateKey(c,key);
} }
void signalFlushedDb(int dbid) { void signalFlushedDb(int dbid, int async) {
touchWatchedKeysOnFlush(dbid); int startdb, enddb;
trackingInvalidateKeysOnFlush(dbid); if (dbid == -1) {
startdb = 0;
enddb = server.dbnum-1;
} else {
startdb = enddb = dbid;
}
for (int j = startdb; j <= enddb; j++) {
touchAllWatchedKeysInDb(&server.db[j], NULL);
}
trackingInvalidateKeysOnFlush(async);
} }
/*----------------------------------------------------------------------------- /*-----------------------------------------------------------------------------
...@@ -593,7 +604,7 @@ int getFlushCommandFlags(client *c, int *flags) { ...@@ -593,7 +604,7 @@ int getFlushCommandFlags(client *c, int *flags) {
/* Parse the optional ASYNC option. */ /* Parse the optional ASYNC option. */
if (c->argc > 1) { if (c->argc > 1) {
if (c->argc > 2 || strcasecmp(c->argv[1]->ptr,"async")) { if (c->argc > 2 || strcasecmp(c->argv[1]->ptr,"async")) {
addReply(c,shared.syntaxerr); addReplyErrorObject(c,shared.syntaxerr);
return C_ERR; return C_ERR;
} }
*flags = EMPTYDB_ASYNC; *flags = EMPTYDB_ASYNC;
...@@ -606,7 +617,7 @@ int getFlushCommandFlags(client *c, int *flags) { ...@@ -606,7 +617,7 @@ int getFlushCommandFlags(client *c, int *flags) {
/* Flushes the whole server data set. */ /* Flushes the whole server data set. */
void flushAllDataAndResetRDB(int flags) { void flushAllDataAndResetRDB(int flags) {
server.dirty += emptyDb(-1,flags,NULL); server.dirty += emptyDb(-1,flags,NULL);
if (server.rdb_child_pid != -1) killRDBChild(); if (server.child_type == CHILD_TYPE_RDB) killRDBChild();
if (server.saveparamslen > 0) { if (server.saveparamslen > 0) {
/* Normally rdbSave() will reset dirty, but we don't want this here /* Normally rdbSave() will reset dirty, but we don't want this here
* as otherwise FLUSHALL will not be replicated nor put into the AOF. */ * as otherwise FLUSHALL will not be replicated nor put into the AOF. */
...@@ -616,6 +627,9 @@ void flushAllDataAndResetRDB(int flags) { ...@@ -616,6 +627,9 @@ void flushAllDataAndResetRDB(int flags) {
rdbSave(server.rdb_filename,rsiptr); rdbSave(server.rdb_filename,rsiptr);
server.dirty = saved_dirty; server.dirty = saved_dirty;
} }
/* Without that extra dirty++, when db was already empty, FLUSHALL will
* not be replicated nor put into the AOF. */
server.dirty++; server.dirty++;
#if defined(USE_JEMALLOC) #if defined(USE_JEMALLOC)
/* jemalloc 5 doesn't release pages back to the OS when there's no traffic. /* jemalloc 5 doesn't release pages back to the OS when there's no traffic.
...@@ -839,7 +853,7 @@ void scanGenericCommand(client *c, robj *o, unsigned long cursor) { ...@@ -839,7 +853,7 @@ void scanGenericCommand(client *c, robj *o, unsigned long cursor) {
} }
if (count < 1) { if (count < 1) {
addReply(c,shared.syntaxerr); addReplyErrorObject(c,shared.syntaxerr);
goto cleanup; goto cleanup;
} }
...@@ -858,7 +872,7 @@ void scanGenericCommand(client *c, robj *o, unsigned long cursor) { ...@@ -858,7 +872,7 @@ void scanGenericCommand(client *c, robj *o, unsigned long cursor) {
typename = c->argv[i+1]->ptr; typename = c->argv[i+1]->ptr;
i+= 2; i+= 2;
} else { } else {
addReply(c,shared.syntaxerr); addReplyErrorObject(c,shared.syntaxerr);
goto cleanup; goto cleanup;
} }
} }
...@@ -1047,7 +1061,7 @@ void shutdownCommand(client *c) { ...@@ -1047,7 +1061,7 @@ void shutdownCommand(client *c) {
int flags = 0; int flags = 0;
if (c->argc > 2) { if (c->argc > 2) {
addReply(c,shared.syntaxerr); addReplyErrorObject(c,shared.syntaxerr);
return; return;
} else if (c->argc == 2) { } else if (c->argc == 2) {
if (!strcasecmp(c->argv[1]->ptr,"nosave")) { if (!strcasecmp(c->argv[1]->ptr,"nosave")) {
...@@ -1055,7 +1069,7 @@ void shutdownCommand(client *c) { ...@@ -1055,7 +1069,7 @@ void shutdownCommand(client *c) {
} else if (!strcasecmp(c->argv[1]->ptr,"save")) { } else if (!strcasecmp(c->argv[1]->ptr,"save")) {
flags |= SHUTDOWN_SAVE; flags |= SHUTDOWN_SAVE;
} else { } else {
addReply(c,shared.syntaxerr); addReplyErrorObject(c,shared.syntaxerr);
return; return;
} }
} }
...@@ -1141,7 +1155,7 @@ void moveCommand(client *c) { ...@@ -1141,7 +1155,7 @@ void moveCommand(client *c) {
/* If the user is moving using as target the same /* If the user is moving using as target the same
* DB as the source DB it is probably an error. */ * DB as the source DB it is probably an error. */
if (src == dst) { if (src == dst) {
addReply(c,shared.sameobjecterr); addReplyErrorObject(c,shared.sameobjecterr);
return; return;
} }
...@@ -1205,7 +1219,7 @@ void copyCommand(client *c) { ...@@ -1205,7 +1219,7 @@ void copyCommand(client *c) {
selectDb(c,srcid); /* Back to the source DB */ selectDb(c,srcid); /* Back to the source DB */
j++; /* Consume additional arg. */ j++; /* Consume additional arg. */
} else { } else {
addReply(c, shared.syntaxerr); addReplyErrorObject(c,shared.syntaxerr);
return; return;
} }
} }
...@@ -1221,7 +1235,7 @@ void copyCommand(client *c) { ...@@ -1221,7 +1235,7 @@ void copyCommand(client *c) {
robj *key = c->argv[1]; robj *key = c->argv[1];
robj *newkey = c->argv[2]; robj *newkey = c->argv[2];
if (src == dst && (sdscmp(key->ptr, newkey->ptr) == 0)) { if (src == dst && (sdscmp(key->ptr, newkey->ptr) == 0)) {
addReply(c,shared.sameobjecterr); addReplyErrorObject(c,shared.sameobjecterr);
return; return;
} }
...@@ -1328,9 +1342,14 @@ int dbSwapDatabases(long id1, long id2) { ...@@ -1328,9 +1342,14 @@ int dbSwapDatabases(long id1, long id2) {
* However normally we only do this check for efficiency reasons * However normally we only do this check for efficiency reasons
* in dbAdd() when a list is created. So here we need to rescan * in dbAdd() when a list is created. So here we need to rescan
* the list of clients blocked on lists and signal lists as ready * the list of clients blocked on lists and signal lists as ready
* if needed. */ * if needed.
*
* Also the swapdb should make transaction fail if there is any
* client watching keys */
scanDatabaseForReadyLists(db1); scanDatabaseForReadyLists(db1);
touchAllWatchedKeysInDb(db1, db2);
scanDatabaseForReadyLists(db2); scanDatabaseForReadyLists(db2);
touchAllWatchedKeysInDb(db2, db1);
return C_OK; return C_OK;
} }
...@@ -1501,6 +1520,12 @@ int expireIfNeeded(redisDb *db, robj *key) { ...@@ -1501,6 +1520,12 @@ int expireIfNeeded(redisDb *db, robj *key) {
* we think the key is expired at this time. */ * we think the key is expired at this time. */
if (server.masterhost != NULL) return 1; if (server.masterhost != NULL) return 1;
/* If clients are paused, we keep the current dataset constant,
* but return to the client what we believe is the right state. Typically,
* at the end of the pause we will properly expire the key OR we will
* have failed over and the new primary will send us the expire. */
if (checkClientPauseTimeoutAndReturnIfPaused()) return 1;
/* Delete the key */ /* Delete the key */
server.stat_expiredkeys++; server.stat_expiredkeys++;
propagateExpire(db,key,server.lazyfree_lazy_expire); propagateExpire(db,key,server.lazyfree_lazy_expire);
......
...@@ -381,39 +381,88 @@ void mallctl_string(client *c, robj **argv, int argc) { ...@@ -381,39 +381,88 @@ void mallctl_string(client *c, robj **argv, int argc) {
void debugCommand(client *c) { void debugCommand(client *c) {
if (c->argc == 2 && !strcasecmp(c->argv[1]->ptr,"help")) { if (c->argc == 2 && !strcasecmp(c->argv[1]->ptr,"help")) {
const char *help[] = { const char *help[] = {
"ASSERT -- Crash by assertion failed.", "AOF-FLUSH-SLEEP <microsec>",
"CHANGE-REPL-ID -- Change the replication IDs of the instance. Dangerous, should be used only for testing the replication subsystem.", " Server will sleep before flushing the AOF, this is used for testing.",
"CRASH-AND-RECOVER <milliseconds> -- Hard crash and restart after <milliseconds> delay.", "ASSERT",
"DIGEST -- Output a hex signature representing the current DB content.", " Crash by assertion failed.",
"DIGEST-VALUE <key-1> ... <key-N>-- Output a hex signature of the values of all the specified keys.", "CHANGE-REPL-ID"
"DEBUG PROTOCOL [string|integer|double|bignum|null|array|set|map|attrib|push|verbatim|true|false]", " Change the replication IDs of the instance.",
"ERROR <string> -- Return a Redis protocol error with <string> as message. Useful for clients unit tests to simulate Redis errors.", " Dangerous: should be used only for testing the replication subsystem.",
"LOG <message> -- write message to the server log.", "CONFIG-REWRITE-FORCE-ALL",
"LEAK <string> -- Create a memory leak of the input string.", " Like CONFIG REWRITE but writes all configuration options, including",
"HTSTATS <dbid> -- Return hash table statistics of the specified Redis database.", " keywords not listed in original configuration file or default values.",
"HTSTATS-KEY <key> -- Like htstats but for the hash table stored as key's value.", "CRASH-AND-RECOVER <milliseconds>",
"LOADAOF -- Flush the AOF buffers on disk and reload the AOF in memory.", " Hard crash and restart after a <milliseconds> delay.",
"LUA-ALWAYS-REPLICATE-COMMANDS <0|1> -- Setting it to 1 makes Lua replication defaulting to replicating single commands, without the script having to enable effects replication.", "DIGEST",
"OBJECT <key> -- Show low level info about key and associated value.", " Output a hex signature representing the current DB content.",
"OOM -- Crash the server simulating an out-of-memory error.", "DIGEST-VALUE <key> [<key> ...]",
"PANIC -- Crash the server simulating a panic.", " Output a hex signature of the values of all the specified keys.",
"POPULATE <count> [prefix] [size] -- Create <count> string keys named key:<num>. If a prefix is specified is used instead of the 'key' prefix.", "ERROR <string>",
"RELOAD [MERGE] [NOFLUSH] [NOSAVE] -- Save the RDB on disk and reload it back in memory. By default it will save the RDB file and load it back. With the NOFLUSH option the current database is not removed before loading the new one, but conflicts in keys will kill the server with an exception. When MERGE is used, conflicting keys will be loaded (the key in the loaded RDB file will win). When NOSAVE is used, the server will not save the current dataset in the RDB file before loading. Use DEBUG RELOAD NOSAVE when you want just to load the RDB file you placed in the Redis working directory in order to replace the current dataset in memory. Use DEBUG RELOAD NOSAVE NOFLUSH MERGE when you want to add what is in the current RDB file placed in the Redis current directory, with the current memory content. Use DEBUG RELOAD when you want to verify Redis is able to persist the current dataset in the RDB file, flush the memory content, and load it back.", " Return a Redis protocol error with <string> as message. Useful for clients",
"RESTART -- Graceful restart: save config, db, restart.", " unit tests to simulate Redis errors.",
"SDSLEN <key> -- Show low level SDS string info representing key and value.", "LOG <message>",
"SEGFAULT -- Crash the server with sigsegv.", " Write <message> to the server log.",
"SET-ACTIVE-EXPIRE <0|1> -- Setting it to 0 disables expiring keys in background when they are not accessed (otherwise the Redis behavior). Setting it to 1 reenables back the default.", "HTSTATS <dbid>",
"SET-SKIP-CHECKSUM-VALIDATION <0|1> -- Enables or disables checksum checks for rdb or RESTORE payload.", " Return hash table statistics of the specified Redis database.",
"AOF-FLUSH-SLEEP <microsec> -- Server will sleep before flushing the AOF, this is used for testing", "HTSTATS-KEY <key>",
"SLEEP <seconds> -- Stop the server for <seconds>. Decimals allowed.", " Like HTSTATS but for the hash table stored at <key>'s value.",
"STRUCTSIZE -- Return the size of different Redis core C structures.", "LOADAOF",
"ZIPLIST <key> -- Show low level info about the ziplist encoding.", " Flush the AOF buffers on disk and reload the AOF in memory.",
"STRINGMATCH-TEST -- Run a fuzz tester against the stringmatchlen() function.", "LUA-ALWAYS-REPLICATE-COMMANDS <0|1>",
"CONFIG-REWRITE-FORCE-ALL -- Like CONFIG REWRITE but writes all configuration options, including keywords not listed in original configuration file or default values.", " Setting it to 1 makes Lua replication defaulting to replicating single",
" commands, without the script having to enable effects replication.",
#ifdef USE_JEMALLOC #ifdef USE_JEMALLOC
"MALLCTL <key> [<val>] -- Get or set a malloc tunning integer.", "MALLCTL <key> [<val>]",
"MALLCTL-STR <key> [<val>] -- Get or set a malloc tunning string.", " Get or set a malloc tuning integer.",
"MALLCTL-STR <key> [<val>]",
" Get or set a malloc tuning string.",
#endif #endif
"OBJECT <key>",
" Show low level info about `key` and associated value.",
"OOM",
" Crash the server simulating an out-of-memory error.",
"PANIC",
" Crash the server simulating a panic.",
"POPULATE <count> [<prefix>] [<size>]",
" Create <count> string keys named key:<num>. If <prefix> is specified then",
" it is used instead of the 'key' prefix.",
"DEBUG PROTOCOL <type>",
" Reply with a test value of the specified type. <type> can be: string,",
" integer, double, bignum, null, array, set, map, attrib, push, verbatim,",
" true, false.",
"RELOAD [option ...]",
" Save the RDB on disk and reload it back to memory. Valid <option> values:",
" * MERGE: conflicting keys will be loaded from RDB.",
" * NOFLUSH: the existing database will not be removed before load, but",
" conflicting keys will generate an exception and kill the server."
" * NOSAVE: the database will be loaded from an existing RDB file.",
" Examples:",
" * DEBUG RELOAD: verify that the server is able to persist, flsuh and reload",
" the database.",
" * DEBUG RELOAD NOSAVE: replace the current database with the contents of an",
" existing RDB file.",
" * DEBUG RELOAD NOSAVE NOFLUSH MERGE: add the contents of an existing RDB",
" file to the database.",
"RESTART",
" Graceful restart: save config, db, restart.",
"SDSLEN <key>",
" Show low level SDS string info representing `key` and value.",
"SEGFAULT",
" Crash the server with sigsegv.",
"SET-ACTIVE-EXPIRE <0|1>",
" Setting it to 0 disables expiring keys in background when they are not",
" accessed (otherwise the Redis behavior). Setting it to 1 reenables back the",
" default.",
"SET-SKIP-CHECKSUM-VALIDATION <0|1>",
" Enables or disables checksum checks for RDB files and RESTORE's payload.",
"SLEEP <seconds>",
" Stop the server for <seconds>. Decimals allowed.",
"STRINGMATCH-TEST",
" Run a fuzz tester against the stringmatchlen() function.",
"STRUCTSIZE",
" Return the size of different Redis core C structures.",
"ZIPLIST <key>",
" Show low level info about the ziplist encoding of <key>.",
NULL NULL
}; };
addReplyHelp(c, help); addReplyHelp(c, help);
...@@ -474,7 +523,7 @@ NULL ...@@ -474,7 +523,7 @@ NULL
rdbSaveInfo rsi, *rsiptr; rdbSaveInfo rsi, *rsiptr;
rsiptr = rdbPopulateSaveInfo(&rsi); rsiptr = rdbPopulateSaveInfo(&rsi);
if (rdbSave(server.rdb_filename,rsiptr) != C_OK) { if (rdbSave(server.rdb_filename,rsiptr) != C_OK) {
addReply(c,shared.err); addReplyErrorObject(c,shared.err);
return; return;
} }
} }
...@@ -500,7 +549,7 @@ NULL ...@@ -500,7 +549,7 @@ NULL
int ret = loadAppendOnlyFile(server.aof_filename); int ret = loadAppendOnlyFile(server.aof_filename);
unprotectClient(c); unprotectClient(c);
if (ret != C_OK) { if (ret != C_OK) {
addReply(c,shared.err); addReplyErrorObject(c,shared.err);
return; return;
} }
server.dirty = 0; /* Prevent AOF / replication */ server.dirty = 0; /* Prevent AOF / replication */
...@@ -512,7 +561,7 @@ NULL ...@@ -512,7 +561,7 @@ NULL
char *strenc; char *strenc;
if ((de = dictFind(c->db->dict,c->argv[2]->ptr)) == NULL) { if ((de = dictFind(c->db->dict,c->argv[2]->ptr)) == NULL) {
addReply(c,shared.nokeyerr); addReplyErrorObject(c,shared.nokeyerr);
return; return;
} }
val = dictGetVal(de); val = dictGetVal(de);
...@@ -564,7 +613,7 @@ NULL ...@@ -564,7 +613,7 @@ NULL
sds key; sds key;
if ((de = dictFind(c->db->dict,c->argv[2]->ptr)) == NULL) { if ((de = dictFind(c->db->dict,c->argv[2]->ptr)) == NULL) {
addReply(c,shared.nokeyerr); addReplyErrorObject(c,shared.nokeyerr);
return; return;
} }
val = dictGetVal(de); val = dictGetVal(de);
...@@ -1569,7 +1618,7 @@ void logCurrentClient(void) { ...@@ -1569,7 +1618,7 @@ void logCurrentClient(void) {
} }
/* Check if the first argument, usually a key, is found inside the /* Check if the first argument, usually a key, is found inside the
* selected DB, and if so print info about the associated object. */ * selected DB, and if so print info about the associated object. */
if (cc->argc >= 1) { if (cc->argc > 1) {
robj *val, *key; robj *val, *key;
dictEntry *de; dictEntry *de;
...@@ -1808,7 +1857,7 @@ void bugReportEnd(int killViaSignal, int sig) { ...@@ -1808,7 +1857,7 @@ void bugReportEnd(int killViaSignal, int sig) {
); );
/* free(messages); Don't call free() with possibly corrupted memory. */ /* free(messages); Don't call free() with possibly corrupted memory. */
if (server.daemonize && server.supervised == 0) unlink(server.pidfile); if (server.daemonize && server.supervised == 0 && server.pidfile) unlink(server.pidfile);
if (!killViaSignal) { if (!killViaSignal) {
if (server.use_exit_on_panic) if (server.use_exit_on_panic)
......
...@@ -38,4 +38,4 @@ ...@@ -38,4 +38,4 @@
fprintf(fp,__VA_ARGS__); \ fprintf(fp,__VA_ARGS__); \
fprintf(fp,"\n"); \ fprintf(fp,"\n"); \
fclose(fp); \ fclose(fp); \
} while (0); } while (0)
...@@ -58,7 +58,6 @@ void* activeDefragAlloc(void *ptr) { ...@@ -58,7 +58,6 @@ void* activeDefragAlloc(void *ptr) {
void *newptr; void *newptr;
if(!je_get_defrag_hint(ptr)) { if(!je_get_defrag_hint(ptr)) {
server.stat_active_defrag_misses++; server.stat_active_defrag_misses++;
size = zmalloc_size(ptr);
return NULL; return NULL;
} }
/* move this allocation to a new allocation. /* move this allocation to a new allocation.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment