- 24 May, 2012 2 commits
-
-
antirez authored
During the AOF rewrite process, the parent process needs to accumulate the new writes in an in-memory buffer: when the child will terminate the AOF rewriting process this buffer (that ist the difference between the dataset when the rewrite was started, and the current dataset) is flushed to the new AOF file. We used to implement this buffer using an sds.c string, but sds.c has a 2GB limit. Sometimes the dataset can be big enough, the amount of writes so high, and the rewrite process slow enough that we overflow the 2GB limit, causing a crash, documented on github by issue #504. In order to prevent this from happening, this commit introduces a new system to accumulate writes, implemented by a linked list of blocks of 10 MB each, so that we also avoid paying the reallocation cost. Note that theoretically modern operating systems may implement realloc() simply as a remaping of the old pages, thus with very good performances, see for instance the mremap() syscall on Linux. However this is not always true, and jemalloc by default avoids doing this because there are issues with the current implementation of mremap(). For this reason we are using a linked list of blocks instead of a single block that gets reallocated again and again. The changes in this commit lacks testing, that will be performed before merging into the unstable branch. This fix will not enter 2.4 because it is too invasive. However 2.4 will log a warning when the AOF rewrite buffer is near to the 2GB limit.
-
antirez authored
The user @jokea noticed that the following line of code into replication.c made little sense: addReplySds(slave,sdsempty()); Investigating a bit I found that this was introduced by commit 6208b3a7 three years ago in the early stages of Redis. The code apparently is not useful at all, so I'm removing it. This change will not be backported into 2.4 so that in the rare case this should introduce a bug, we'll have a chance to detect it into the development branch. However following the code path it seems like the code is not useful at all, so the risk is truly small.
-
- 23 May, 2012 2 commits
-
-
jokea authored
-
antirez authored
Weeks ago trying to fix an harmless GCC warning I introduced a bug in the ziplist-encoded implementations of sorted sets. The bug completely broke zuiNext() iterator, that is used in the ZINTERSTORE and ZUNIONSTORE implementation, so those two commands are no longer reliable starting from Redis version 2.4.12 and latest 2.6.0-RC releases. This commit fixes the problem and adds a regression test.
-
- 22 May, 2012 1 commit
-
-
antirez authored
Due to a change in the format of the bug report in case of crash of failed assertion the test suite was no longer able to properly log it. Instead just a protocol error was logged by the Redis TCL client that provided no clue about the actual problem. This commit resolves the issue by logging everything from the first line of the log including the string REDIS BUG REPORT, till the end of the file.
-
- 21 May, 2012 1 commit
-
-
antirez authored
This makes the code more readable, it is still not the case to split the file itself into three different files, but the logical separation improves the readability especially since new commits are going to introduce an additional section.
-
- 20 May, 2012 1 commit
-
-
antirez authored
The list of things to do is since long time in two places: 1) Github issues. 2) I've a private TOOD list of random ideas, what makes sense is later moved to github issues. So github is anyway the true source of things to do.
-
- 16 May, 2012 2 commits
-
-
antirez authored
In the commit upgrading jemalloc to version 3.0.0 I added the old version of Jemalloc in the 'jemalloc.orig' directory for an error. This commit removes the not useful version of jemalloc.
-
antirez authored
Full changelog here: http://www.canonware.com/cgi-bin/gitweb.cgi?p=jemalloc.git;a=blob_plain;f=ChangeLog;hb=master Notable improvements from the point of view of Redis: 1) Bugfixing. 2) Support for Valgrind. 3) Support for OSX Lion, FreeBSD.
-
- 15 May, 2012 4 commits
-
-
Pieter Noordhuis authored
-
Dave Pacheco authored
-
Dave Pacheco authored
-
Pieter Noordhuis authored
-
- 14 May, 2012 2 commits
-
-
antirez authored
redis-cli.c uses the time() function to seed the PRNG, but time.h was not included. This was not noticed since sys/time.h is included and was enough in most systems (but not correct). With Ubuntu 12.04 GCC generates a warning that made us aware of the issue.
-
antirez authored
activeExpireCycle() can consume no more than a few milliseconds per iteration. This commit improves the precision of the check for the time elapsed in two ways: 1) We check every 16 iterations instead of the main loop instead of 256. 2) We reset iterations at the start of the function and not every time we switch to the next database, so the check is correctly performed every 16 iterations.
-
- 13 May, 2012 2 commits
-
-
antirez authored
A previous commit introduced REDIS_HZ define that changes the frequency of calls to the serverCron() Redis function. This commit improves different related things: 1) Software watchdog: now the minimal period can be set according to REDIS_HZ. The minimal period is two times the timer period, that is: (1000/REDIS_HZ)*2 milliseconds 2) The incremental rehashing is now performed in the expires dictionary as well. 3) The activeExpireCycle() function was improved in different ways: - Now it checks if it already used too much time using microseconds instead of milliseconds for better precision. - The time limit is now calculated correctly, in the previous version the division was performed before of the multiplication resulting in a timelimit of 0 if HZ was big enough. - Databases with less than 1% of buckets fill in the hash table are skipped, because getting random keys is too expensive in this condition. 4) tryResizeHashTables() is now called at every timer call, we need to match the number of calls we do to the expired keys colleciton cycle. 5) REDIS_HZ was raised to 100.
-
antirez authored
Redis uses a function called serverCron() that is very similar to the timer interrupt of an operating system. This function is used to handle a number of asynchronous things, like active expired keys collection, clients timeouts, update of statistics, things related to the cluster and replication, triggering of BGSAVE and AOF rewrite process, and so forth. In the past the timer was called 1 time per second. At some point it was raised to 10 times per second, but it still was fixed and could not be changed even at compile time, because different functions called from serverCron() assumed a given fixed frequency. This commmit makes the frequency configurable, so that it is simpler to pick a good tradeoff between overhead of this function (that is usually very small) and the responsiveness of Redis during a few critical circumstances where a lot of work is done inside the timer. An example of such a critical condition is mass-expire of a lot of keys in the same second. Up to a given percentage of CPU time is used to perform expired keys collection per expire cylce. Now changing the REDIS_HZ macro it is possible to do less work but more times per second in order to block the server for less time. If this patch will work well in our tests it will enter Redis 2.6-final.
-
- 11 May, 2012 5 commits
-
-
antirez authored
-
antirez authored
If a large amonut of keys are all expiring about at the same time, the "active" expired keys collection cycle used to block as far as the percentage of already expired keys was >= 25% of the total population of keys with an expire set. This could block the server even for many seconds in order to reclaim memory ASAP. The new algorithm uses at max a small amount of milliseconds per cycle, even if this means reclaiming the memory less promptly it also means a more responsive server.
-
antirez authored
-
antirez authored
-
antirez authored
-
- 09 May, 2012 2 commits
- 07 May, 2012 1 commit
-
-
antirez authored
-
- 06 May, 2012 1 commit
-
-
Pieter Noordhuis authored
Because of the introduction of new integer encoding types for ziplists in the 2.6 tree, the same integer value may have a different encoding in different versions of the ziplist implementation. This means that the encoding can NOT be used as a fast path in comparing integers.
-
- 04 May, 2012 2 commits
-
-
Salvatore Sanfilippo authored
fix several bugs of init.d scripts
-
quiver authored
- PIDFILE environ variable was not properly retrieved - chkconfig command failed
-
- 02 May, 2012 4 commits
-
-
antirez authored
The new implementation start reading / writing before blocking with aeWait(), likely the descriptor can accept writes or has buffered data inside and we can go faster, otherwise we get an error and wait. This change has effects on speed but also on correctness: on socket errors when we perform non blocking connect(2) write is performed ASAP and the error is returned ASAP before waiting. So the practical effect is that now a Redis slave is more available if it can not connect to the master, previously the slave continued to block on syncWrite() trying to send SYNC, and serving commands very slowly.
-
antirez authored
-
antirez authored
We used to reply -ERR ... message ..., now the reply is instead -MASTERDOWN ... message ... so that it can be distinguished easily by the other error conditions.
-
antirez authored
-
- 01 May, 2012 2 commits
-
-
Pieter Noordhuis authored
Every matched key in a KEYS call is checked for expiration. When the key is set to expire, the call to `getExpire` will assert that the key also exists in the main dictionary. This in turn causes a rehashing step to be executed. Rehashing a dictionary when there is an iterator active may result in the iterator emitting duplicate entries, or not emitting some entries at all. By using a safe iterator, the rehash step is omitted.
-
Harmen authored
-
- 30 Apr, 2012 1 commit
-
-
antirez authored
-
- 29 Apr, 2012 1 commit
-
-
antirez authored
-
- 27 Apr, 2012 3 commits
-
-
antirez authored
-
antirez authored
-
antirez authored
An user reported a crash with Redis scripting (see issue #480 on github), inspection of the kindly provided strack trace showed that server.lua_caller was probably set to NULL. The stack trace also slowed that the call to the hook was originating from a point where we just used to set/get a few global variables in the Lua state. What was happening is that we did not set the timeout hook selectively only when the user script was called. Now we set it more selectively, specifically only in the context of the lua_pcall() call, and make sure to remove the hook when the call returns. Otherwise the hook can get called in random contexts every time we do something with the Lua state.
-
- 26 Apr, 2012 1 commit
-
-
antirez authored
A previous commit removed -g -rdynamic -ggdb as LDFLAGS, not allowing Redis to produce a stack trace wth symbol names on crash. This commit fixes the issue.
-