http.DefaultClient has no timeout
Its Timeout field is the zero value, meaning wait forever. Set a Client.Timeout and pass a context as well: the context bounds the whole pagination run, the client timeout bounds each request.
Instagram API / Code samples / Go
Every endpoint shares one response shape, so one envelope struct with a json.RawMessage data field covers all sixteen and you decode the payload into whatever type the call actually returns.
First call
Copy this, set your key, and you have a working integration. Everything below is the same request with more of the edge cases handled.
Quickstart
package main
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/url"
"os"
"time"
)
type Envelope struct {
Success bool `json:"success"`
CreditsCharged int `json:"credits_charged"`
CreditsRemaining int `json:"credits_remaining"`
ProcessingTimeMs int `json:"processing_time_ms"`
RequestedAt string `json:"requested_at"`
Query map[string]any `json:"query"`
Data json.RawMessage `json:"data"`
Error *APIError `json:"error"`
}
type APIError struct {
Code string `json:"code"`
Message string `json:"message"`
}
type Profile struct {
Handle string `json:"handle"`
FollowerCount int64 `json:"follower_count"`
IsVerified bool `json:"is_verified"`
}
func main() {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
q := url.Values{"handle": {"natgeo"}}
req, err := http.NewRequestWithContext(ctx, http.MethodGet,
"https://api.socialscrape.dev/v1/instagram/profile?"+q.Encode(), nil)
if err != nil {
panic(err)
}
req.Header.Set("x-api-key", os.Getenv("INSCRAPE_KEY"))
res, err := http.DefaultClient.Do(req)
if err != nil {
panic(err)
}
defer res.Body.Close()
var env Envelope
if err := json.NewDecoder(res.Body).Decode(&env); err != nil {
panic(err)
}
if res.StatusCode != http.StatusOK {
// A non-200 body always carries error.code, and credits_charged is 0.
fmt.Println(res.StatusCode, env.Error.Code, env.Error.Message)
os.Exit(1)
}
var profile Profile
if err := json.Unmarshal(env.Data, &profile); err != nil {
panic(err)
}
fmt.Println(profile.Handle, profile.FollowerCount, profile.IsVerified)
fmt.Println("charged", env.CreditsCharged, "remaining", env.CreditsRemaining, "requested", env.RequestedAt)
}Pagination
Every list endpoint returns a cursor. Stop when it is absent, not after a fixed number of pages — the page size is not a contract.
Go — cursor loop
package main
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/url"
"os"
"time"
)
type Envelope struct {
Success bool `json:"success"`
CreditsCharged int `json:"credits_charged"`
CreditsRemaining int `json:"credits_remaining"`
ProcessingTimeMs int `json:"processing_time_ms"`
RequestedAt string `json:"requested_at"`
Query map[string]any `json:"query"`
Data json.RawMessage `json:"data"`
Error *APIError `json:"error"`
}
type APIError struct {
Code string `json:"code"`
Message string `json:"message"`
}
type PostsPage struct {
Handle string `json:"handle"`
NextCursor string `json:"next_cursor"`
Posts []Post `json:"posts"`
}
type Post struct {
Shortcode string `json:"shortcode"`
Type string `json:"type"`
LikeCount int64 `json:"like_count"`
CommentCount int64 `json:"comment_count"`
TakenAt string `json:"taken_at"`
}
func fetchPosts(ctx context.Context, client *http.Client, handle, cursor string) (Envelope, PostsPage, error) {
var env Envelope
var page PostsPage
q := url.Values{"handle": {handle}, "limit": {"50"}}
if cursor != "" {
q.Set("cursor", cursor)
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet,
"https://api.socialscrape.dev/v1/instagram/posts?"+q.Encode(), nil)
if err != nil {
return env, page, err
}
req.Header.Set("x-api-key", os.Getenv("INSCRAPE_KEY"))
res, err := client.Do(req)
if err != nil {
return env, page, err
}
defer res.Body.Close()
if err := json.NewDecoder(res.Body).Decode(&env); err != nil {
return env, page, err
}
if res.StatusCode != http.StatusOK {
code := "UNKNOWN"
if env.Error != nil {
code = env.Error.Code
}
return env, page, fmt.Errorf("%d %s", res.StatusCode, code)
}
err = json.Unmarshal(env.Data, &page)
return env, page, err
}
func main() {
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Minute)
defer cancel()
client := &http.Client{Timeout: 30 * time.Second}
var all []Post
cursor := ""
spent := 0
for pages := 0; pages < 10; pages++ {
env, page, err := fetchPosts(ctx, client, "natgeo", cursor)
if err != nil {
panic(err)
}
spent += env.CreditsCharged
all = append(all, page.Posts...)
cursor = page.NextCursor
if cursor == "" {
break
}
}
fmt.Println(len(all), "posts across", spent, "credits")
}Errors
Only HTTP 200 is charged, so a retry after a 500 costs you nothing except time. A retry after a 400 costs you nothing and fixes nothing.
Go — error handling
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"net/url"
"os"
"time"
)
type Envelope struct {
Success bool `json:"success"`
CreditsCharged int `json:"credits_charged"`
CreditsRemaining int `json:"credits_remaining"`
ProcessingTimeMs int `json:"processing_time_ms"`
RequestedAt string `json:"requested_at"`
Query map[string]any `json:"query"`
Data json.RawMessage `json:"data"`
Error *APIError `json:"error"`
}
type APIError struct {
Code string `json:"code"`
Message string `json:"message"`
}
type StatusError struct {
Status int
Code string
Msg string
}
func (e *StatusError) Error() string {
return fmt.Sprintf("%d %s: %s", e.Status, e.Code, e.Msg)
}
// Retryable reports whether another attempt can succeed. Only a 200 is ever
// charged, so a retry costs credits only when it eventually returns 200.
func (e *StatusError) Retryable() bool { return e.Status >= 500 }
func call(ctx context.Context, client *http.Client, path string, q url.Values) (Envelope, error) {
var env Envelope
req, err := http.NewRequestWithContext(ctx, http.MethodGet,
"https://api.socialscrape.dev/v1"+path+"?"+q.Encode(), nil)
if err != nil {
return env, err
}
req.Header.Set("x-api-key", os.Getenv("INSCRAPE_KEY"))
res, err := client.Do(req)
if err != nil {
return env, err
}
defer res.Body.Close()
if err := json.NewDecoder(res.Body).Decode(&env); err != nil {
return env, err
}
if res.StatusCode != http.StatusOK {
code, msg := "UNKNOWN", ""
if env.Error != nil {
code, msg = env.Error.Code, env.Error.Message
}
return env, &StatusError{Status: res.StatusCode, Code: code, Msg: msg}
}
return env, nil
}
func callWithRetry(ctx context.Context, client *http.Client, path string, q url.Values) (Envelope, error) {
var env Envelope
var err error
for attempt := 0; attempt < 3; attempt++ {
env, err = call(ctx, client, path, q)
if err == nil {
return env, nil
}
var se *StatusError
if !errors.As(err, &se) || !se.Retryable() {
return env, err
}
time.Sleep(time.Duration(1<<attempt) * time.Second)
}
return env, err
}
func main() {
ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
defer cancel()
env, err := callWithRetry(ctx, &http.Client{Timeout: 30 * time.Second},
"/instagram/profile", url.Values{"handle": {"natgeo"}})
var se *StatusError
switch {
case errors.As(err, &se) && se.Code == "ENDPOINT_NOT_FOUND":
fmt.Println("API route not found")
case errors.As(err, &se) && se.Code == "INSUFFICIENT_CREDITS":
fmt.Println("out of credits")
case err != nil:
panic(err)
default:
var data struct { Profile json.RawMessage `json:"profile"` }
if err := json.Unmarshal(env.Data, &data); err != nil { panic(err) }
if string(data.Profile) == "null" {
fmt.Println("account not found; credits charged:", env.CreditsCharged)
}
fmt.Println("credits left:", env.CreditsRemaining)
}
}Gotchas
Its Timeout field is the zero value, meaning wait forever. Set a Client.Timeout and pass a context as well: the context bounds the whole pagination run, the client timeout bounds each request.
The envelope never changes across the sixteen endpoints; only data does. Decoding data lazily lets you share the credit and request handling and keep the per-endpoint types small.
A 200 response has no error key, so a non-pointer struct would silently give you an empty Code and hide the branch. Check res.StatusCode before dereferencing env.Error.
Do not build the query with fmt.Sprintf. Cursors contain characters that must be percent-encoded, and a hand-built string breaks on page two rather than page one, which makes it annoying to spot.
FAQ
No. The code above is the client library, and it will not go stale when we add an endpoint. If you want types for every payload, generate them from https://socialscrape.dev/openapi.json.
A worker pool of ten to twenty goroutines reading from a channel of handles. Higher concurrency does not make the upstream faster; it only makes your credit spend arrive sooner and your failures arrive together.
There is no published per-second limit. Under a sustained burst you will see 504 UPSTREAM_TIMEOUT and 500 SCRAPE_FAILED rather than a 429, and neither is charged. Back off on 5xx and the queue drains.
No. Pagination is cursor-based over ordinary JSON responses; there is no streaming or webhook delivery. Loop on the returned cursor until it is empty. Paginated responses return data.next_cursor followed by data.has_next_page.
No. There is nowhere to supply one, and the service does not use Instagram accounts at all. Confirmed private accounts return 200 with available profile details and are charged at the endpoint rate.
Other languages
Every endpoint these samples call is documented at /docs/instagram.