1. 11 Sep, 2018 1 commit
  2. 04 Sep, 2018 1 commit
  3. 03 Sep, 2018 4 commits
  4. 31 Aug, 2018 3 commits
    • antirez's avatar
      After slave Lua script leaves busy state, re-process the master buffer. · 7fa49391
      antirez authored
      Technically speaking we don't really need to put the master client in
      the clients that need to be processed, since in practice the PING
      commands from the master will take care, however it is conceptually more
      sane to do so.
      7fa49391
    • antirez's avatar
      While the slave is busy, just accumulate master input. · 9ab91b8c
      antirez authored
      Processing command from the master while the slave is in busy state is
      not correct, however we cannot, also, just reply -BUSY to the
      replication stream commands from the master. The correct solution is to
      stop processing data from the master, but just accumulate the stream
      into the buffers and resume the processing later.
      
      Related to #5297.
      9ab91b8c
    • zhaozhao.zz's avatar
      networking: fix unexpected negative or zero readlen · dce7cefb
      zhaozhao.zz authored
      To avoid copying buffers to create a large Redis Object which
      exceeding PROTO_IOBUF_LEN 32KB, we just read the remaining data
      we need, which may less than PROTO_IOBUF_LEN. But the remaining
      len may be zero, if the bulklen+2 equals sdslen(c->querybuf),
      in client pause context.
      
      For example:
      
      Time1:
      
      python
      >>> import os, socket
      >>> server="127.0.0.1"
      >>> port=6379
      >>> data1="*3\r\n$3\r\nset\r\n$1\r\na\r\n$33000\r\n"
      >>> data2="".join("x" for _ in range(33000)) + "\r\n"
      >>> data3="\n\n"
      >>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
      >>> s.settimeout(10)
      >>> s.connect((server, port))
      >>> s.send(data1)
      28
      
      Time2:
      
      redis-cli client pause 10000
      
      Time3:
      
      >>> s.send(data2)
      33002
      >>> s.send(data3)
      2
      >>> s.send(data3)
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
      socket.error: [Errno 104] Connection reset by peer
      
      To fix that, we should check if remaining is greater than zero.
      dce7cefb
  5. 23 Aug, 2018 1 commit
    • zhaozhao.zz's avatar
      networking: make setProtocolError simple and clear · f2ad89a3
      zhaozhao.zz authored
      Function setProtocolError just records proctocol error
      details in server log, set client as CLIENT_CLOSE_AFTER_REPLY.
      It doesn't care about querybuf sdsrange, because we
      will do it after procotol parsing.
      f2ad89a3
  6. 14 Aug, 2018 2 commits
  7. 13 Aug, 2018 1 commit
    • zhaozhao.zz's avatar
      pipeline: do not sdsrange querybuf unless all commands processed · 14c4ddb5
      zhaozhao.zz authored
      This is an optimization for processing pipeline, we discussed a
      problem in issue #5229: clients may be paused if we apply `CLIENT
      PAUSE` command, and then querybuf may grow too large, the cost of
      memmove in sdsrange after parsing a completed command will be
      horrible. The optimization is that parsing all commands in queyrbuf
      , after that we can just call sdsrange only once.
      14c4ddb5
  8. 18 Jul, 2018 2 commits
  9. 17 Jul, 2018 2 commits
    • antirez's avatar
      Panic when we are sending an error to our master/slave. · afc7e08a
      antirez authored
      Related to #5135, see discussion there.
      afc7e08a
    • Oran Agra's avatar
      fix rare replication stream corruption with disk-based replication · d5559898
      Oran Agra authored
      The slave sends \n keepalive messages to the master while parsing the rdb,
      and later sends REPLCONF ACK once a second. rarely, the master recives both
      a linefeed char and a REPLCONF in the same read, \n*3\r\n$8\r\nREPLCONF\r\n...
      and it tries to trim two chars (\r\n) from the query buffer,
      trimming the '*' from *3\r\n$8\r\nREPLCONF\r\n...
      
      then the master tries to process a command starting with '3' and replies to
      the slave a bunch of -ERR and one +OK.
      although the slave silently ignores these (prints a log message), this corrupts
      the replication offset at the slave since the slave increases the replication
      offset, and the master did not.
      
      other than the fix in processInlineBuffer, i did several other improvments
      while hunting this very rare bug.
      
      - when redis replies with "unknown command" it includes a portion of the
        arguments, not just the command name. so it would be easier to understand
        what was recived, in my case, on the slave side,  it was -ERR, but
        the "arguments" were the interesting part (containing info on the error).
      - about a year ago i added code in addReplyErrorLength to print the error to
        the log in case of a reply to master (since this string isn't actually
        trasmitted to the master), now changed that block to print a similar log
        message to indicate an error being sent from the master to the slave.
        note that the slave is marked as CLIENT_SLAVE only after PSYNC was received,
        so this will not cause any harm for REPLCONF, and will only indicate problems
        that are gonna corrupt the replication stream anyway.
      - two places were c->reply was emptied, and i wanted to reset sentlen
        this is a precaution (i did not actually see such a problem), since a
        non-zero sentlen will cause corruption to be transmitted on the socket.
      d5559898
  10. 16 Jul, 2018 2 commits
    • antirez's avatar
      Hopefully improve commenting of #5126. · f9c84d6d
      antirez authored
      Reading the PR gave me the opportunity to better specify what the code
      was doing in places where I was not immediately sure about what was
      going on. Moreover I documented the structure in server.h so that people
      reading the header file will immediately understand what the structure
      is useful for.
      f9c84d6d
    • Oran Agra's avatar
      slave buffers were wasteful and incorrectly counted causing eviction · bf680b6f
      Oran Agra authored
      A) slave buffers didn't count internal fragmentation and sds unused space,
         this caused them to induce eviction although we didn't mean for it.
      
      B) slave buffers were consuming about twice the memory of what they actually needed.
      - this was mainly due to sdsMakeRoomFor growing to twice as much as needed each time
        but networking.c not storing more than 16k (partially fixed recently in 237a38737).
      - besides it wasn't able to store half of the new string into one buffer and the
        other half into the next (so the above mentioned fix helped mainly for small items).
      - lastly, the sds buffers had up to 30% internal fragmentation that was wasted,
        consumed but not used.
      
      C) inefficient performance due to starting from a small string and reallocing many times.
      
      what i changed:
      - creating dedicated buffers for reply list, counting their size with zmalloc_size
      - when creating a new reply node from, preallocate it to at least 16k.
      - when appending a new reply to the buffer, first fill all the unused space of the
        previous node before starting a new one.
      
      other changes:
      - expose mem_not_counted_for_evict info field for the benefit of the test suite
      - add a test to make sure slave buffers are counted correctly and that they don't cause eviction
      bf680b6f
  11. 09 Jul, 2018 3 commits
  12. 02 Jul, 2018 1 commit
  13. 29 Jun, 2018 1 commit
  14. 28 Jun, 2018 2 commits
  15. 27 Jun, 2018 6 commits
  16. 13 Jun, 2018 1 commit
  17. 07 Jun, 2018 1 commit
  18. 22 Mar, 2018 2 commits
  19. 15 Mar, 2018 2 commits
  20. 28 Feb, 2018 1 commit
  21. 27 Feb, 2018 1 commit
    • antirez's avatar
      AOF: fix a bug that may prevent proper fsyncing when fsync=always. · 75987431
      antirez authored
      In case the write handler is already installed, it could happen that we
      serve the reply of a query in the same event loop cycle we received it,
      preventing beforeSleep() from guaranteeing that we do the AOF fsync
      before sending the reply to the client.
      
      The AE_BARRIER mechanism, introduced in a previous commit, prevents this
      problem. This commit makes actual use of this new feature to fix the
      bug.
      75987431