Fix Lua C API violation on lua msgpack lib. (#9832)
msgpack lib missed using lua_checkstack and so on rare cases overflow the stack by at most 2 elements. This is a violation of the Lua C API. Notice that Lua allocates additional 5 more elements on top of lua->stack_last so Redis does not access an invalid memory. But it is an API violation and we should avoid it. This PR also added a new Lua compilation option. The new option can be enable using environment variable called LUA_DEBUG. If set to `yes` (by default `no`), Lua will be compiled without optimizations and with debug symbols (`-O0 -g`). In addition, in this new mode, Lua will be compiled with the `-DLUA_USE_APICHECK` flag that enables extended Lua C API validations. In addition, set LUA_DEBUG=yes on daily valgrind flow so we will be able to catch Lua C API violations in the future.
Please register or sign in to comment