Rate limiting and retry logic for PubChemAPI #15

New Issue

2026-04-12T16:47:22-04:00

aalexmmaldonado commented

2026-04-12 16:47:22 -04:00

(Migrated from github.com)

PubChem allows up to 5 requests per second and 400 per minute. The current make_request method fires with no throttling and raises RuntimeError on any non-200 status, including the 503 PubChem returns when you've been rate-limited. Dissociation constants alone have 40+ pages. Hit that endpoint without a delay, and you will get throttled.

Scope

Add a configurable delay between requests, default 200ms (5 req/s). A class-level _last_request_time with a sleep in make_request is enough.

Add retry logic for HTTP 503 (rate limited) and 504 (gateway timeout): retry up to 3 times with exponential backoff (1s, 2s, 4s), then raise the existing RuntimeError. Log retries at the WARNING level.

Test with a mock session that returns 503 twice, then 200.

Definition of done

Rate-limited requests retry automatically. A minimum delay is enforced between consecutive requests. Tested with mock sessions.

PubChem allows up to 5 requests per second and 400 per minute. The current `make_request` method fires with no throttling and raises `RuntimeError` on any non-200 status, including the 503 PubChem returns when you've been rate-limited. Dissociation constants alone have 40+ pages. Hit that endpoint without a delay, and you will get throttled. ### Scope Add a configurable delay between requests, default 200ms (5 req/s). A class-level `_last_request_time` with a sleep in `make_request` is enough. Add retry logic for HTTP 503 (rate limited) and 504 (gateway timeout): retry up to 3 times with exponential backoff (1s, 2s, 4s), then raise the existing `RuntimeError`. Log retries at the WARNING level. Test with a mock session that returns 503 twice, then 200. ### Definition of done Rate-limited requests retry automatically. A minimum delay is enforced between consecutive requests. Tested with mock sessions.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: data/pcdigitizer#15