• antirez's avatar
    Draft #1 of a new expired keys collection algorithm. · ab782615
    antirez authored
    The main idea here is that when we are no longer to expire keys at the
    rate the are created, we can't block more in the normal expire cycle as
    this would result in too big latency spikes.
    
    For this reason the commit introduces a "fast" expire cycle that does
    not run for more than 1 millisecond but is called in the beforeSleep()
    hook of the event loop, so much more often, and with a frequency bound
    to the frequency of executed commnads.
    
    The fast expire cycle is only called when the standard expiration
    algorithm runs out of time, that is, consumed more than
    REDIS_EXPIRELOOKUPS_TIME_PERC of CPU in a given cycle without being able
    to take the number of already expired keys that are yet not collected
    to a number smaller than 25% of the number of keys.
    
    You can test this commit with different loads, but a simple way is to
    use the following:
    
    Extreme load with pipelining:
    
    redis-benchmark -r 100000000 -n 100000000  \
            -P 32 set ele:rand:000000000000 foo ex 2
    
    Remove the -P32 in order to avoid the pipelining for a more real-world
    load.
    
    In another terminal tab you can monitor the Redis behavior with:
    
    redis-cli -i 0.1 -r -1 info keyspace
    
    and
    
    redis-cli --latency-history
    
    Note: this commit will make Redis printing a lot of debug messages, it
    is not a good idea to use it in production.
    ab782615
redis.h 51.1 KB