requests has no default timeout
Without timeout= a call can hang until the socket dies, which in a worker means a stuck job rather than an error you can see. Thirty seconds is comfortable; our upstream fetch aborts at 25 and returns 504 before that.
Instagram API / Code samples / Python
A focused code reference for developers who have chosen the API and want working requests calls. The separate production-pipeline guide covers batching, storage and cost control.
First call
Copy this, set your key, and you have a working integration. Everything below is the same request with more of the edge cases handled.
Install
pip install requestsQuickstart
import os
import requests
BASE = "https://api.socialscrape.dev/v1"
KEY = os.environ["INSCRAPE_KEY"]
res = requests.get(
f"{BASE}/instagram/profile",
params={"handle": "natgeo"},
headers={"x-api-key": KEY},
timeout=30,
)
body = res.json()
if res.status_code != 200:
err = body["error"]
raise SystemExit(f"{res.status_code} {err['code']}: {err['message']}")
data = body["data"]
print(data["handle"], data["follower_count"], data["is_verified"])
print(
"charged", body["credits_charged"],
"remaining", body["credits_remaining"],
"requested", body["requested_at"],
"ms", body["processing_time_ms"],
)Pagination
Every list endpoint returns a cursor. Stop when it is absent, not after a fixed number of pages — the page size is not a contract.
Python — cursor loop
import os
import requests
BASE = "https://api.socialscrape.dev/v1"
session = requests.Session()
session.headers["x-api-key"] = os.environ["INSCRAPE_KEY"]
def all_posts(handle, max_pages=10):
posts = []
spent = 0
cursor = None
pages = 0
while pages < max_pages:
params = {"handle": handle, "limit": 50}
if cursor:
params["cursor"] = cursor
res = session.get(f"{BASE}/instagram/posts", params=params, timeout=30)
body = res.json()
if res.status_code != 200:
raise RuntimeError(f"{res.status_code} {body['error']['code']}")
spent += body["credits_charged"]
posts.extend(body["data"]["posts"])
pages += 1
cursor = body["data"].get("next_cursor")
if not cursor:
break
return posts, spent
items, spent = all_posts("natgeo")
print(len(items), "posts across", spent, "credits")Errors
Only HTTP 200 is charged, so a retry after a 500 costs you nothing except time. A retry after a 400 costs you nothing and fixes nothing.
Python — error handling
import os
import time
import requests
BASE = "https://api.socialscrape.dev/v1"
KEY = os.environ["INSCRAPE_KEY"]
class InScrapeError(RuntimeError):
def __init__(self, status, code, message):
super().__init__(f"{status} {code}: {message}")
self.status = status
self.code = code
def call(path, **params):
# raise_for_status() throws away the body, and the body is where the API
# tells you the error code and that credits_charged is 0. Read it first.
res = requests.get(f"{BASE}{path}", params=params, headers={"x-api-key": KEY}, timeout=30)
body = res.json()
if res.status_code == 200:
return body
err = body.get("error", {})
raise InScrapeError(res.status_code, err.get("code", "UNKNOWN"), err.get("message", ""))
def call_with_retry(path, attempts=3, **params):
for attempt in range(attempts):
try:
return call(path, **params)
except InScrapeError as exc:
# Only 5xx is worth retrying. 4xx will fail the same way forever,
# and only a 200 is ever charged either way.
if exc.status < 500 or attempt == attempts - 1:
raise
time.sleep(2 ** attempt)
try:
body = call_with_retry("/instagram/profile", handle="natgeo")
except InScrapeError as exc:
if exc.code == "INSUFFICIENT_CREDITS":
print("out of credits, top up before retrying")
elif exc.code == "ENDPOINT_NOT_FOUND":
print("API route not found")
elif exc.code == "INVALID_API_KEY":
print("check INSCRAPE_KEY")
else:
raise
else:
if body.get("data", {}).get("profile") is None:
print(body["data"]["profile"], "credits charged:", body["credits_charged"])
else:
print(body["data"]["profile"]["follower_count"], "credits left:", body["credits_remaining"])Gotchas
Without timeout= a call can hang until the socket dies, which in a worker means a stuck job rather than an error you can see. Thirty seconds is comfortable; our upstream fetch aborts at 25 and returns 504 before that.
The envelope is identical, so swapping requests for httpx.AsyncClient is a mechanical change. Cap concurrency with an asyncio.Semaphore; ten to twenty in flight is the operating point /docs/rate-limits recommends. Firing 200 handles at once turns a credit budget into a thundering herd and buys you nothing, because each handle still costs the same.
Keeping requested_at, processing_time_ms and query alongside the payload is what lets you answer when a lookup ran and what triggered it six weeks later. Discarding the envelope is the most common regret in a scraping pipeline.
FAQ
No, and there does not need to be one. The API is plain REST with a single x-api-key header, so a wrapper is the small session object on this page. An SDK would be a versioning liability for both of us.
No. There is no login, no session cookie and no OAuth. Public data only, which is also why confirmed private accounts return 200 with available profile details and are charged at the endpoint rate.
No endpoint returns an engagement rate. You compute it: average likes and comments from /instagram/posts divided by follower_count from /instagram/profile. Two calls, and the formula stays yours.
Read credits_charged. It is the source of truth for what the request cost, and failures before resource use are free.
No. Those are private metrics that only the account owner sees, through Meta's own Graph API with the owner's authorisation. We read what any logged-out visitor can see, and nothing else.
Other languages
Building a batch job rather than one request? Read the production Instagram data pipeline guide for Python.
Every endpoint these samples call is documented at /docs/instagram.