For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.
Connection limits
Limit the in-flight requests and client connections that agentgateway accepts on each gateway port.
Attaches to:
Cap the number of requests and connections that clients can open to the gateway proxy at the same time. Over a limit, the proxy rejects new work immediately and does not queue it. As a result, a traffic spike or a client that opens many connections cannot exhaust the proxy’s memory and file descriptors.
You set both limits in the top-level frontendPolicies section of the configuration file. Each limit counts separately for each gateway port. By default, both limits are unset, so the number of requests and connections is not limited.
| Setting | What the limit counts | What happens over the limit |
|---|---|---|
frontendPolicies.http.maxConcurrentRequests | In-flight HTTP requests, including HTTP/1 requests and HTTP/2 streams. The minimum value is 1. | Agentgateway returns a 503 response with the body request limit exceeded, and increments the agentgateway_requests_shed_total metric. |
frontendPolicies.tcp.maxConnections | Active client connections. The minimum value is 1. | Agentgateway closes the new connection without a response, and increments the agentgateway_downstream_connections_shed_total metric. |
Before you begin
- Install the
agentgatewaybinary. - Make sure that you have
docker, such as by installing Docker Desktop.
Limit requests and connections
Set a request limit and a connection limit in turn, and send more traffic than each limit allows.
Start the httpbin sample app in Docker. The app listens on port 8000 on your machine.
docker run -d --name httpbin -p 8000:8080 mccutchen/go-httpbin:v2.15.0Create a configuration file that allows 2 in-flight requests.
cat > config.yaml <<EOF # yaml-language-server: \$schema=https://agentgateway.dev/schema/config frontendPolicies: http: maxConcurrentRequests: 2 gateways: default: port: 3000 routes: - backends: - host: localhost:8000 EOFRun agentgateway with the configuration file.
agentgateway -f config.yamlIn another terminal, send 3 requests at the same time to the httpbin
/delay/3endpoint, which responds after 3 seconds. The first 2 requests take all the request slots, so agentgateway rejects the third request immediately with a503response.for i in 1 2 3; do curl -s -o /dev/null -w "request $i: %{http_code}\n" http://localhost:3000/delay/3 & sleep 0.2 done; waitExample output:
request 3: 503 request 1: 200 request 2: 200In the configuration file, replace the
frontendPoliciessection with the following section. The new section sets a connection limit of 1 and no request limit. Agentgateway reloads the file when it changes.frontendPolicies: tcp: maxConnections: 1Open a connection with a slow request, and then open a second connection. The first connection takes the only connection slot, so agentgateway closes the second connection without a response.
curl -s -o /dev/null -w "connection 1: %{http_code}\n" http://localhost:3000/delay/3 & sleep 0.5 curl -sS -o /dev/null -w "connection 2: %{http_code}\n" http://localhost:3000/headers waitExample output:
curl: (56) Recv failure: Connection reset by peer connection 2: 000 connection 1: 200Check the metrics that count the rejected requests and connections. The
bindlabel names the gateway port.curl -s localhost:15020/metrics | grep -E '^agentgateway_(requests|downstream_connections)_shed'Example output:
agentgateway_downstream_connections_shed_total{bind="bind/3000"} 1 agentgateway_requests_shed_total{bind="bind/3000"} 1
Cleanup
Stop agentgateway by pressing
Ctrl+Cin the terminal where agentgateway runs.Remove the httpbin container and the configuration file.
docker rm -f httpbin rm config.yaml