ai-proxy use

[nirb28]

Hi Community,

I have a standalone apisix 3.12 instance running on localhost. From the docs, I set this route in apisix.conf:
routes:

• uri: /anything
  upstream:
  scheme: https
  type: roundrobin
  nodes:
  somerandom.com:443: 1
  pass_host: node
  plugins:
  ai-proxy:
  auth:
  header:
  Authorization: Bearer
  model:
  provider: openai
  name: "gpt-3.5-turbo"
  options:
  max_tokens: 512
  temperature: 1.0

However, when running I get a 502 error:
curl "http://127.0.0.1:9080/anything" -X POST -H "X-APISIX-Dynamic-Debug: 1" -H "Content-Type: application/json" -H "Host: api.openai.com:443" -d '{
"messages": [
{
"role": "system",
"content": "You are a computer scientist."
},
{
"role": "user",
"content": "Explain in one sentence what a Turing machine is."
}
]
}'

<title>502 Bad Gateway</title>

502 Bad Gateway

---

openresty

Powered by APISIX.

The debug logs:
2025/04/19 21:30:49 [warn] 54#54: *107187 [lua] debug.lua:190: http_header_filter_phase(): call require("apisix").http_header_filter_phase() args:{} while sending to client, client: 172.20.0.1, server: _, request: "POST /anything HTTP/1.1", upstream: "https://216.239.34.21:443/anything", host: "api.openai.com:443"

2025/04/19 21:30:49 [warn] 54#54: *107187 [lua] debug.lua:198: http_header_filter_phase(): call require("apisix").http_header_filter_phase() return:{} while sending to client, client: 172.20.0.1, server: _, request: "POST /anything HTTP/1.1", upstream: "https://216.239.34.21:443/anything", host: "api.openai.com:443"

2025/04/19 21:30:49 [warn] 54#54: *107187 [lua] debug.lua:190: http_body_filter_phase(): call require("apisix").http_body_filter_phase() args:{} while sending to client, client: 172.20.0.1, server: _, request: "POST /anything HTTP/1.1", upstream: "https://216.239.34.21:443/anything", host: "api.openai.com:443"

2025/04/19 21:30:49 [warn] 54#54: *107187 [lua] debug.lua:198: http_body_filter_phase(): call require("apisix").http_body_filter_phase() return:{} while sending to client, client: 172.20.0.1, server: _, request: "POST /anything HTTP/1.1", upstream: "https://216.239.34.21:443/anything", host: "api.openai.com:443"

2025/04/19 21:30:49 [warn] 54#54: *107187 [lua] debug.lua:190: http_log_phase(): call require("apisix").http_log_phase() args:{} while logging request, client: 172.20.0.1, server: _, request: "POST /anything HTTP/1.1", upstream: "https://216.239.34.21:443/anything", host: "api.openai.com:443"

2025/04/19 21:30:49 [warn] 54#54: *107187 [lua] debug.lua:198: http_log_phase(): call require("apisix").http_log_phase() return:{} while logging request, client: 172.20.0.1, server: _, request: "POST /anything HTTP/1.1", upstream: "https://216.239.34.21:443/anything", host: "api.openai.com:443"

172.20.0.1 - - [19/Apr/2025:21:30:49 +0000] api.openai.com:443 "POST /anything HTTP/1.1" 502 229 0.071 "-" "curl/8.12.1" 216.239.34.21:443 502 0.062 "https://somerandom.com"

Any ideas on what might be wrong?

Thanks,

[dosubot[bot]]

Hello @nirb28! I'm here to help you with your questions and bugs while you wait for a human maintainer. If you need assistance or want to become a contributor, just let me know!

There are two similar closed issues related to the 502 Bad Gateway error when using APISIX:

1. Issue #11726 - No solution found in the comments.
2. Issue #10287 - No solution found in the comments.

Regarding your configuration, the 502 Bad Gateway error might be related to the api-breaker plugin configuration. The api-breaker plugin can return a 502 error if the upstream service is considered unhealthy, which happens if it returns a 500 or 503 status code multiple times within a certain period. You should check the health of the upstream service and ensure it is not frequently returning these error codes. Additionally, you can adjust the unhealthy.failures parameter or modify the unhealthy.http_statuses to exclude certain status codes if appropriate ^[1].

Ensure that your upstream service (OpenAI's GPT-3.5-turbo model) is stable and not frequently returning error status codes that trigger the circuit breaker.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}