• antirez's avatar
    PSYNC2: different improvements to Redis replication. · 2669fb83
    antirez authored
    The gist of the changes is that now, partial resynchronizations between
    slaves and masters (without the need of a full resync with RDB transfer
    and so forth), work in a number of cases when it was impossible
    in the past. For instance:
    
    1. When a slave is promoted to mastrer, the slaves of the old master can
    partially resynchronize with the new master.
    
    2. Chained slalves (slaves of slaves) can be moved to replicate to other
    slaves or the master itsef, without requiring a full resync.
    
    3. The master itself, after being turned into a slave, is able to
    partially resynchronize with the new master, when it joins replication
    again.
    
    In order to obtain this, the following main changes were operated:
    
    * Slaves also take a replication backlog, not just masters.
    
    * Same stream replication for all the slaves and sub slaves. The
    replication stream is identical from the top level master to its slaves
    and is also the same from the slaves to their sub-slaves and so forth.
    This means that if a slave is later promoted to master, it has the
    same replication backlong, and can partially resynchronize with its
    slaves (that were previously slaves of the old master).
    
    * A given replication history is no longer identified by the `runid` of
    a Redis node. There is instead a `replication ID` which changes every
    time the instance has a new history no longer coherent with the past
    one. So, for example, slaves publish the same replication history of
    their master, however when they are turned into masters, they publish
    a new replication ID, but still remember the old ID, so that they are
    able to partially resynchronize with slaves of the old master (up to a
    given offset).
    
    * The replication protocol was slightly modified so that a new extended
    +CONTINUE reply from the master is able to inform the slave of a
    replication ID change.
    
    * REPLCONF CAPA is used in order to notify masters that a slave is able
    to understand the new +CONTINUE reply.
    
    * The RDB file was extended with an auxiliary field that is able to
    select a given DB after loading in the slave, so that the slave can
    continue receiving the replication stream from the point it was
    disconnected without requiring the master to insert "SELECT" statements.
    This is useful in order to guarantee the "same stream" property, because
    the slave must be able to accumulate an identical backlog.
    
    * Slave pings to sub-slaves are now sent in a special form, when the
    top-level master is disconnected, in order to don't interfer with the
    replication stream. We just use out of band "\n" bytes as in other parts
    of the Redis protocol.
    
    An old design document is available here:
    
    https://gist.github.com/antirez/ae068f95c0d084891305
    
    However the implementation is not identical to the description because
    during the work to implement it, different changes were needed in order
    to make things working well.
    2669fb83
rdb.h 5.87 KB