| Files | Last commit | Last update |
|---|---|---|
remove authors | 6 years ago | |
remove authors | 6 years ago | |
Bound wire-declared sizes and decompressed output during parsing (#3476) Several parsers that handle client-controlled input trusted sizes declared on the wire without bounding them, which could make a connection consume far more memory than the message itself: - gzip/zlib/snappy decompression had no output cap: -max_body_size is only checked against the compressed bytes, so a small body could decompress to tens of GiB. Add -max_decompressed_body_size (default 32x -max_body_size, 0 means use the default) and enforce it in all three decompressors. The output cap is applied via a local ZeroCopyInputStream wrapper so it builds against every protobuf version CI uses (>= 3.5.1); the wrapper clips oversized blocks without double-backing-up the wrapped stream. - AMF string readers resized the output buffer to the declared length before checking how many bytes were actually available; read the string in bounded chunks instead. - RTMP chunk headers may re-declare a message while a pr- RTMP chunk headers may re-declare a message while a pr- RTMP chunk headers may re-deed - RTMP chunk headers may re-declare a message while a pr- RTMP ody- RTMP chunk headers may re-declare a message while a pr- RTwir- RTMP chunk headers may re-declare a message while a pr- cou- RTMP chunk headers may re-declare a message while a pr- RTMP ch r- RTMP chunk headers may re-declare a message while a pr- RTMPvalues instead of CHECK-fatal. | 2 months ago | |
Small FlatMap optimization with default initialization (#2620) * Small FlatMap optimization * Support optional | 1 year ago | |
remove authors | 6 years ago | |
Reject malformed mcpack input instead of CHECK-fatal in parser (#3576) * Reject malformed mcpack input instead of CHECK-fatal in parser The mcpack2pb parser used CHECK(false) to handle malformed input in unbox(), the object/array/isoarray iterators and the value conversion functions. CHECK(false) logs at FATAL level and aborts the process when -crash_on_fatal_log is on, so a single malformed nshead+mcpack request could terminate a server and drop all in-flight requests (CWE-617). Replace the wire-controlled CHECK(false) sites with LOG(ERROR) plus the existing error propagation (set_bad()/return 0, which the generated parsing code checks), and mark the stream bad in unbox() and the truncated as_string()/as_binary() paths so failed parses are reported by stream()->good() as before. The serializer and code-generator CHECKs are untouched since they are not driven by network input. * Avoid wire-controlled eager resize in as_string/as_binary Follow-up to the CHECK(false) hardening: the claimed value size of string/binary fields comes from the wire and can be close to UINT32_MAX. resize(size) before reading allocates that amount up front, so a tiny request could trigger an uncaught std::bad_alloc/std::length_error and still terminate the process. Read in bounded chunks and grow the output only for bytes that are actually present; a short read marks the stream bad as before. * Reject truncated primitive reads in cut_packed_pod The value-returning InputStream::cut_packed_pod<T>() left its result uninitialized when cutn() read fewer than sizeof(T) bytes, which is reachable when the enclosing object declares enough bytes but the real input ends inside a primitive value. The indeterminate value was then consumed or logged (UB), and the parse continued as if the input were complete. Zero-initialize the result, return a zero value and mark the stream bad on a short read, so the generated code fails the parse. The pointer version is unchanged: its callers already check the returned size. * Validate the trailing NUL when reading mcpack strings as_string() skipped the trailing byte with popn(1), which returns 0 on an exhausted stream and leaves it good. A string whose content was present but whose stream ended exactly before the required trailing '\0' (or ended with a non-NUL byte) was therefore accepted as valid. Read the terminator and reject the input when it is missing or not a NUL, marking the stream bad and clearing the output. * Avoid double copy of valid string/binary payload cut_bytes_to_string() copied every byte twice on valid input: once from the input into a stack buffer and once into the destination string. Grow the destination by at most one 8 KiB chunk at a time and cut directly into it, restoring the single-copy behavior of the original resize+cutn path while keeping the bounded allocation. --------- Co-authored-by: brpc-oncall <brpc-oncall@localhost> | 1 month ago | |
Reject malformed mcpack input instead of CHECK-fatal in parser (#3576) * Reject malformed mcpack input instead of CHECK-fatal in parser The mcpack2pb parser used CHECK(false) to handle malformed input in unbox(), the object/array/isoarray iterators and the value conversion functions. CHECK(false) logs at FATAL level and aborts the process when -crash_on_fatal_log is on, so a single malformed nshead+mcpack request could terminate a server and drop all in-flight requests (CWE-617). Replace the wire-controlled CHECK(false) sites with LOG(ERROR) plus the existing error propagation (set_bad()/return 0, which the generated parsing code checks), and mark the stream bad in unbox() and the truncated as_string()/as_binary() paths so failed parses are reported by stream()->good() as before. The serializer and code-generator CHECKs are untouched since they are not driven by network input. * Avoid wire-controlled eager resize in as_string/as_binary Follow-up to the CHECK(false) hardening: the claimed value size of string/binary fields comes from the wire and can be close to UINT32_MAX. resize(size) before reading allocates that amount up front, so a tiny request could trigger an uncaught std::bad_alloc/std::length_error and still terminate the process. Read in bounded chunks and grow the output only for bytes that are actually present; a short read marks the stream bad as before. * Reject truncated primitive reads in cut_packed_pod The value-returning InputStream::cut_packed_pod<T>() left its result uninitialized when cutn() read fewer than sizeof(T) bytes, which is reachable when the enclosing object declares enough bytes but the real input ends inside a primitive value. The indeterminate value was then consumed or logged (UB), and the parse continued as if the input were complete. Zero-initialize the result, return a zero value and mark the stream bad on a short read, so the generated code fails the parse. The pointer version is unchanged: its callers already check the returned size. * Validate the trailing NUL when reading mcpack strings as_string() skipped the trailing byte with popn(1), which returns 0 on an exhausted stream and leaves it good. A string whose content was present but whose stream ended exactly before the required trailing '\0' (or ended with a non-NUL byte) was therefore accepted as valid. Read the terminator and reject the input when it is missing or not a NUL, marking the stream bad and clearing the output. * Avoid double copy of valid string/binary payload cut_bytes_to_string() copied every byte twice on valid input: once from the input into a stack buffer and once into the destination string. Grow the destination by at most one 8 KiB chunk at a time and cut directly into it, restoring the single-copy behavior of the original resize+cutn path while keeping the bounded allocation. --------- Co-authored-by: brpc-oncall <brpc-oncall@localhost> | 1 month ago | |
Limit recursion depth of mcpack2pb parser (#3499) * Limit recursion depth of mcpack2pb parser The mcpack2pb parser recurses once per nesting level of the incoming object/array. A message with an excessively deep recursive structure makes the recursion grow unbounded and the stack overflow, crashing the process. The serializer already enforces MAX_DEPTH=128, but the parse path never checked it. Thread the nesting depth through UnparsedValue and the iterators, and fail the parse once the depth exceeds MAX_DEPTH, mirroring the serializer. Deeply nested but legitimate messages keep working. * Refine depth-limit tests and comments after review Make the depth semantics in parser.h consistent with the implementation (UnparsedValue::depth counts the containers the value is nested in, iterators add one), document that the 3-arg set() intentionally keeps the depth, and trim the unit test: parse a payload a few levels beyond MAX_DEPTH instead of 16384 levels, which is equally effective, much faster and avoids O(depth^2) payload construction. Also destroy the pthread attribute on every path. * Align iterator depth default and fix test attribute cleanup Default the stream-based ObjectIterator/ArrayIterator depth to 1 so the top-level container always has depth 1 whether built from a value (depth+1) or directly from a stream, matching the documented semantics and keeping the MAX_DEPTH guard effective for all constructor paths. In the unit test, destroy the pthread attribute on a single shared path right after create (when it is still valid and no longer needed), instead of unconditionally calling pthread_attr_destroy which would be undefined behavior if pthread_attr_init failed. | 2 months ago | |
remove authors | 6 years ago | |
remove authors | 6 years ago | |
remove authors | 6 years ago |
| Files | Last commit | Last update |
|---|---|---|
| 6 years ago | ||
| 6 years ago | ||
| 2 months ago | ||
| 1 year ago | ||
| 6 years ago | ||
| 1 month ago | ||
| 1 month ago | ||
| 2 months ago | ||
| 6 years ago | ||
| 6 years ago | ||
| 6 years ago |