Narf Discovers Critical Vulnerabilities in Cesanta Mongoose HTTP Server

Narf's investigation of HTTP parsers for the DARPA SafeDocs program led us to uncover three vulnerabilities in Cesanta Mongoose. This effort was part of a larger initiative in which we analyze HTTP parsers in proxies, servers, load balancers, CDNs, and clients to characterize mismatches in protocol implementations.

The first bug was an easily-triggered denial-of-service attack that can be triggered remotely. This bug was fixed in the Mongoose 7.10 release. This bug was hidden in plain sight despite Mongoose's participation in the OSS-Fuzz project. To trigger this vulnerability, a request must have a Content-Length header with a value exactly equal to the request's negated length. It's unlikely that random mutations will arrive at such a specific value. That said, it's fairly likely that OSS-Fuzz has generated fuzzing inputs with negative Content-Lengths, but since the fuzzer has no understanding of the HTTP grammar and does not compare program output to that of other HTTP servers, it would not be aware that it triggered a bug. Format-Aware Differential Fuzzing is needed to compare program outputs while generating syntactically well-formed inputs.

The second vulnerability was a request smuggling vulnerability where Mongoose does not correctly interpret the Content-Length header in HTTP requests. This bug was fixed in the master branch, but releases before Mongoose 6.18 remain vulnerable. Finally, the third vulnerability is another request smuggling bug that can be triggered by injecting an empty header. This bug was fixed in the Mongoose 7.11 release.

All three vulnerabilities identified by the Narf team point to inadequate input validation in Mongoose's HTTP implementation. These examples demonstrate that HTTP is a data format of sufficient complexity to benefit from principled parser generator toolkits developed for the SafeDocs program, such as Daedalus and Hammer.

Background

Cesanta Mongoose is a network library that supports many network protocols and platforms. It is deployed by companies such as Siemens, Schneider Electric, Broadcom, Bosch, Google, Samsung, Qualcomm, and Caterpillar, primarily to run web interfaces on embedded computers. There are presently over 2000 public IPs running Mongoose servers.¹ Although most of these servers are configured to not respond with the version number in the HTTP responses, several servers run a default configuration where they explicitly disclose the Mongoose version.

HTTP requests are largely understood to have three primary components: the request line, the set of headers, and the body. While this abstract view is generally straightforward to understand, it turns out that correctly delimiting these partitions is challenging because of the allowed internal complexity of each, especially given the permitted flexibility in the set of legal delimiter values. The request line states which resource is being requested, the headers are key-value pairs that specify additional request information, and the body is a sequence of bytes that constitutes the data being sent with the request. Below is an example HTTP POST request.

POST / HTTP/1.1
Host: narfindustries.com
Connection: Keep-Alive
accept-encoding: gzip
user-agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/114.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
accept-language: en-US,en;q=0.5
Content-Type: application/x-www-form-urlencoded
Content-Length: 27

field1=value1&field2=value2

This apparently human-readable example request is misleading even though it is emblematic of many HTTP requests and consistent with the informal expectation that HTTP requests are generally legible to humans and well-structured in their appearance. However, the data ingest code of most web servers sees merely a flat contiguous sequence of almost arbitrary byte values.

The Content-Length header specifies the length of an HTTP message's body. A valid Content-Length header has a value consisting of one or more ASCII digits. For example, when a server receives a request with a Content-Length header with the value 100, it must expect that the request body consists of 100 octets. Any HTTP request is untrusted data that an attacker controls. The attacker could always supply no request body or a body that is much larger, invalidating the assertion that the length of the body must be 100 octets.

HTTP pipelining is a feature of HTTP/1.1 that allows clients to send multiple HTTP requests on the same TCP connection. Servers that support pipelining can then respond to each of the received requests in sequence. Pipelining improves the performance of an HTTP server by reducing the memory and CPU burden of allocating new sockets and input buffers and reduces the time taken to establish TCP handshakes. Because pipelined HTTP requests share a connection, and there is no delimiter between messages, an HTTP parser must be able to accurately determine the boundaries between pipelined requests based on Content-Length alone.²

Discovery

These bugs were discovered by configuring a collection of HTTP servers to respond to every request in order to observe the malformations that are accepted by popular HTTP parsers. During manual testing, we prefixed Content-Length values with -, and noticed that Mongoose responds in three different ways: sometimes without error, sometimes responds twice, and sometimes closes the connection. By binary searching to the values at the borderlines of the ranges that cause these behaviors, we arrived at an input that causes denial of service.

CVE-2023-34188: Denial of Service

Mongoose employs an HTTP shotgun parser that does not properly validate HTTP Content-Length headers, and can be fooled into endlessly reading and re-reading the same request in a busy loop. Once it has entered this state, the server cannot respond to further requests and has effectively been taken down. Mongoose accepts Content-Length values with a - prefix, and treats them as negative. This behavior occurs because Mongoose used the function mg_to64(struct mg_str str) to convert a string to a 64-bit signed integer. The fix for this CVE was to add a new function to only work with the size_t datatype and remove support for the mg_to64 function.

When parsing a (potentially pipelined) request, Mongoose determines where the request ends by reading its headers until it encounters the end-of-headers delimiter, then adding the Content-Length value to the offset of the end of the delimiter. This is intended to skip forward over the message body, but if the Content-Length is negative, Mongoose will instead skip backward. Thus, if the Content-Length value is the negation of the length of the entire request, the parser will skip back to the beginning of the request and parse it again. Generating the exploit for this vulnerability is trivial.

# Clone the Mongoose repository
git clone https://github.com/cesanta/mongoose

# Check out a vulnerable release, build it, and start the server
cd mongoose && git checkout 7.9
make &

# Send the payload
printf "$EXPLOIT_PAYLOAD" | nc localhost 8000

# In Python 3.10.6
>>> m = hashlib.sha256()
>>> m.update(b"Narf6/26/2023" + EXPLOIT_PAYLOAD)
>>> m.digest()
b'\xf8\x9a\xaa\x95\xc5R8}\xf2"W\xdc\xba\xf6\xccP\x1aKx\xc5\x9e\xe7\xcd\x04\x0eJ^\xe9\xcf\xac\\)'

After that, the following message should be endlessly printed to the screen: 5bc0ae39 2 main.c:32:cb GET / 200 1946

Note: We will release the exploit payload at a later time.

Timeline

• April 27, 2023: We disclose the vulnerability to Cesanta.
• April 28, 2023: Cesanta responds, saying that they'll investigate.
• May 17, 2023: Cesanta asks that we wait 30 days before making a security advisory.
• May 18, 2023: Cesanta confirms that they can reproduce the attack, and releases Mongoose 7.10, which patches the vulnerability.
• June 17, 2023: The 30-day period ends.

HTTP Request Smuggling via incorrect Content-Length parsing

Often, HTTP servers are configured with a proxy server that is public-facing. The proxy accepts HTTP requests and forwards them to the backend HTTP servers. In pipelined requests, request smuggling attacks could arise from mismatches between how the proxy and the backend servers interpret where various requests begin. The PortSwigger team documented request smuggling attacks by mixing Transfer-Encoding and Content-Length in the same HTTP requests.³

Let us consider the following HTTP requests:

Message 1

GET / HTTP/1.1
Host: A
Content-Length: 27
Content-Lengthh: 0

GET / HTTP/1.1
Host: A

Message 2

GET / HTTP/1.1
Host: A
Content-Length: 27
Content-Length: 0

GET / HTTP/1.1
Host: A

The only difference between the two messages is in a trailing "h" character in Message 1. However, Mongoose interprets the Content-Lengthh as Content-Length because it performed a prefix match and considered the size of the payload to be 0. Proxy servers tend to leave the Content-Lengthh header as is since it may be a custom header field.

A proxy server, such as H2O, accepts the first Content-Length value, and rewrites the request to contain only one Content-Length header.⁴ Hence, passing Message 2 via H2O to Mongoose does not have the same effect as passing Message 1. Sending both messages directly to Mongoose has the same effect--it interprets the Content-Length to be 0, but then ignores it entirely.

Timeline

• April 27, 2023: We disclose the vulnerability to Cesanta.
• April 28, 2023: Cesanta responds, saying that they'll investigate.
• May 17, 2023: Cesanta says that given that the bug is fixed in the master branch, they will not fix.

Truncated Headers Causing Request Smuggling

We found that we could trick the Mongoose server into dropping headers by injecting an empty header.

GET / HTTP/1.1
Host: whatever
:
Content-Length: 1

The line with the : is an empty header that does not contain any key or value. In the above example, the Content-Length header is ignored by Mongoose. The correct way to handle this request is to either ignore only the empty header, or to treat the entire request as malformed and return a 400 response (Bad Request).

Note: HAProxy exhibited a similar error that has now been fixed: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-25725.

Timeline

• April 27, 2023: We disclose the vulnerability to Cesanta.
• April 28, 2023: Cesanta responds, saying that they'll investigate.
• May 17, 2023: Cesanta asks us to open an issue on GitHub since they do not consider this as a threat.
• June 26, 2023: Narf Industries reported the bug publicly at GitHub, at the request of the Cesanta team.
• July 3, 2023: Cesanta releases Mongoose 7.11, fixing the reported bug.

Disclaimer: This material is based in part upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Contract No. HR001119C0073. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Defense Advanced Research Projects Agency (DARPA).