BadgeprintingEvent registration · badge automation

Polling Eventbrite Web APIs Without Rate Limiting

Symptom Manifestation & Diagnostic Baseline

Event operations teams deploying automated badge printing pipelines consistently encounter a cascading failure signature: badge generation queues stall while host processes spike to 100% CPU, and downstream Eventbrite API endpoints return 429 Too Many Requests. The operational impact compounds rapidly: on-site check-in latency increases linearly, attendee records diverge from ticketing ledgers, and payment reconciliation gaps widen as unprinted badges block downstream settlement workflows. Diagnostic traces across failed deployments reveal three concurrent failure modes:

  1. Linear polling loops that ignore X-RateLimit-Remaining and X-RateLimit-Reset headers, treating the API as stateless.
  2. Cursor drift where pagination tokens are discarded on process restart, causing identical dataset re-fetches.
  3. Missing idempotency guards that trigger duplicate badge generation when transient network errors force retry logic.

When webhook delivery degrades due to Eventbrite platform latency or regional network partitioning, fallback polling defaults to aggressive fixed-interval requests. This behavior directly violates established Form API Polling Strategies and introduces measurable drift in downstream Registration Ingestion & Payment Reconciliation pipelines.

Root Cause Analysis

The failure is architectural, not infrastructural. Stateless ingestion engines decouple request velocity from API capacity constraints. Without persistent state, every restart resets to updated_after=epoch, forcing full dataset scans. Without token-bucket enforcement, retry storms exhaust the tenant request budget within seconds. Without cryptographic deduplication, network partitions cause duplicate processing, corrupting badge queues and triggering reconciliation mismatches. The combination of unbounded retry logic and missing rate-capacity awareness guarantees 429 exhaustion under any webhook degradation scenario.

Resolution Architecture

Eliminating rate-limit exhaustion requires shifting from naive interval polling to a stateful, adaptive ingestion model. The architecture enforces four non-negotiable constraints:

  1. Persistent Cursor Tracking: updated_after timestamps and continuation tokens are written to a durable SQLite store before each API call, surviving process crashes and deployments.
  2. Token-Bucket Rate Limiting: Request dispatch aligns with Eventbrite’s documented tier limits. The bucket drains on successful requests and refills based on X-RateLimit-Reset headers. When capacity hits zero, dispatch blocks until the window resets.
  3. Exponential Backoff with Jitter: Transient 5xx or 429 responses trigger truncated exponential backoff (base=2s, max=60s) with uniform jitter to prevent thundering herd synchronization.
  4. Strict Idempotency: Every attendee payload is hashed via SHA-256 before processing. A bounded in-memory LRU cache prevents duplicate badge generation without unbounded memory growth.

Memory and performance boundaries are enforced at the code level: SQLite handles durable state with WAL mode for low-lock contention, the in-memory hash cache caps at 50,000 entries to prevent OOM, and pagination streams results row-by-row rather than buffering full responses.

Production-Grade Python Implementation

The following engine requires only the Python standard library and requests. It implements cursor persistence, adaptive scheduling, token-bucket rate limiting, and cryptographic deduplication.

PYTHON
import os
import time
import json
import hashlib
import sqlite3
import random
import requests
from collections import OrderedDict
from typing import Optional, Dict, Any

# Configuration aligned with Eventbrite standard tier limits
EVENTBRITE_API_BASE = "https://www.eventbriteapi.com/v3"
DEFAULT_RATE_LIMIT = 100  # requests per minute
DEFAULT_RESET_WINDOW = 60  # seconds
MAX_HASH_CACHE = 50_000   # Bounded memory footprint

class EventbritePoller:
    def __init__(self, api_token: str, event_id: str, db_path: str = "poller_state.db"):
        self.api_token = api_token
        self.event_id = event_id
        self.session = requests.Session()
        self.session.headers.update({
            "Authorization": f"Bearer {api_token}",
            "Accept": "application/json",
            "User-Agent": "EventOps-BadgePipeline/1.0"
        })
        
        # Durable state store (WAL mode for concurrent read/write safety)
        self.db_path = db_path
        self._init_db()
        
        # Token bucket state
        self.tokens = DEFAULT_RATE_LIMIT
        self.last_refill = time.time()
        
        # Bounded idempotency cache (LRU eviction)
        self.processed_hashes = OrderedDict()

    def _init_db(self):
        with sqlite3.connect(self.db_path) as conn:
            conn.execute("PRAGMA journal_mode=WAL;")
            conn.execute("""
                CREATE TABLE IF NOT EXISTS poll_state (
                    key TEXT PRIMARY KEY,
                    value TEXT
                )
            """)
            conn.commit()

    def _get_state(self, key: str) -> Optional[str]:
        with sqlite3.connect(self.db_path) as conn:
            cur = conn.execute("SELECT value FROM poll_state WHERE key=?", (key,))
            row = cur.fetchone()
            return row[0] if row else None

    def _set_state(self, key: str, value: str):
        with sqlite3.connect(self.db_path) as conn:
            conn.execute("INSERT OR REPLACE INTO poll_state (key, value) VALUES (?, ?)", (key, value))
            conn.commit()

    def _hash_payload(self, payload: Dict[str, Any]) -> str:
        # Deterministic hash of attendee ID + updated timestamp
        canonical = json.dumps({
            "id": payload.get("id"),
            "updated": payload.get("updated")
        }, sort_keys=True)
        return hashlib.sha256(canonical.encode()).hexdigest()

    def _is_duplicate(self, payload_hash: str) -> bool:
        if payload_hash in self.processed_hashes:
            return True
        if len(self.processed_hashes) >= MAX_HASH_CACHE:
            self.processed_hashes.popitem(last=False)  # Evict oldest
        self.processed_hashes[payload_hash] = True
        return False

    def _wait_for_token(self):
        """Token-bucket rate limiter with header-aware refill."""
        while self.tokens <= 0:
            now = time.time()
            elapsed = now - self.last_refill
            if elapsed >= DEFAULT_RESET_WINDOW:
                self.tokens = DEFAULT_RATE_LIMIT
                self.last_refill = now
            else:
                time.sleep(1)

    def _consume_token(self, headers: Dict[str, str]):
        remaining = headers.get("X-RateLimit-Remaining")
        if remaining is not None:
            self.tokens = max(0, int(remaining))
        else:
            self.tokens -= 1

    def _calculate_backoff(self, attempt: int) -> float:
        """Truncated exponential backoff with uniform jitter."""
        delay = min(2 ** attempt, 60)
        jitter = random.uniform(0, delay * 0.25)
        return delay + jitter

    def fetch_attendees(self):
        """Stateful, rate-limited, idempotent attendee ingestion loop."""
        continuation = self._get_state("continuation_token")
        updated_after = self._get_state("updated_after") or "2020-01-01T00:00:00Z"
        attempt = 0

        while True:
            self._wait_for_token()
            
            params = {
                "status": "attending",
                "updated_after": updated_after,
                "expand": "ticket_class"
            }
            if continuation:
                params["continuation"] = continuation

            try:
                resp = self.session.get(
                    f"{EVENTBRITE_API_BASE}/events/{self.event_id}/attendees/",
                    params=params,
                    timeout=15
                )
                resp.raise_for_status()
                attempt = 0  # Reset on success

                # Consume rate limit token
                self._consume_token(resp.headers)

                data = resp.json()
                attendees = data.get("attendees", [])
                
                # Process stream
                for attendee in attendees:
                    payload_hash = self._hash_payload(attendee)
                    if self._is_duplicate(payload_hash):
                        continue
                    self._process_attendee(attendee)
                    # Advance monotonic cursor
                    if attendee.get("updated") and attendee["updated"] > updated_after:
                        updated_after = attendee["updated"]

                # Persist state immediately after batch
                self._set_state("updated_after", updated_after)
                
                # Handle pagination
                pagination = data.get("pagination", {})
                continuation = pagination.get("continuation")
                if continuation:
                    self._set_state("continuation_token", continuation)
                    # Adaptive sleep: scale with remaining rate capacity
                    sleep_time = max(0.5, (1 - (self.tokens / DEFAULT_RATE_LIMIT)) * 5)
                    time.sleep(sleep_time)
                    continue
                else:
                    # End of dataset reached, reset pagination cursor
                    self._set_state("continuation_token", "")
                    print("Poll cycle complete. Waiting for next window.")
                    time.sleep(DEFAULT_RESET_WINDOW)
                    updated_after = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
                    self._set_state("updated_after", updated_after)
                    continue

            except requests.exceptions.HTTPError as e:
                if e.response.status_code == 429:
                    retry_after = int(e.response.headers.get("Retry-After", 10))
                    print(f"Rate limited. Backing off for {retry_after}s")
                    time.sleep(retry_after)
                    continue
                elif e.response.status_code >= 500:
                    delay = self._calculate_backoff(attempt)
                    print(f"Server error {e.response.status_code}. Backing off {delay:.1f}s")
                    time.sleep(delay)
                    attempt += 1
                    continue
                raise
            except requests.exceptions.RequestException as e:
                delay = self._calculate_backoff(attempt)
                print(f"Network error. Backing off {delay:.1f}s")
                time.sleep(delay)
                attempt += 1
                continue

    def _process_attendee(self, attendee: Dict[str, Any]):
        """Idempotent badge generation / downstream dispatch hook."""
        # Replace with actual badge print queue push or reconciliation sync
        pass

if __name__ == "__main__":
    TOKEN = os.getenv("EVENTBRITE_API_TOKEN")
    EVENT_ID = os.getenv("EVENTBRITE_EVENT_ID")
    if not TOKEN or not EVENT_ID:
        raise EnvironmentError("Set EVENTBRITE_API_TOKEN and EVENTBRITE_EVENT_ID")
    
    poller = EventbritePoller(TOKEN, EVENT_ID)
    poller.fetch_attendees()

Incident Response & Rollback Procedures

When rate-limit exhaustion or badge queue corruption occurs, execute the following sequence to restore stability within 5 minutes:

  1. Immediate Triage:
  • Pause the poller process (SIGSTOP or container scale-to-zero).
  • Verify poller_state.db integrity: sqlite3 poller_state.db "PRAGMA integrity_check;"
  • Drain in-memory queues to prevent duplicate badge dispatch.
  1. State Recovery:
  • Restore poller_state.db from the last known-good backup if cursor drift is detected.
  • Flush processed_hashes by restarting the service (LRU cache is ephemeral by design).
  1. Rollback Execution:
  • Revert to the previous stable container image or deployment revision.
  • Clear any stuck continuation_token values that point to expired Eventbrite cursors: sqlite3 poller_state.db "DELETE FROM poll_state WHERE key='continuation_token';"
  • Restart with EVENTBRITE_API_TOKEN and EVENTBRITE_EVENT_ID validated.
  1. Post-Incident Validation:
  • Monitor X-RateLimit-Remaining headers via structured logs. Alert if < 10 for >60 seconds.
  • Verify badge print queue depth matches updated_after progression.
  • Confirm payment reconciliation gap closes within one full poll cycle.

For deeper backoff implementation standards, reference the AWS Architecture Blog on Exponential Backoff and Jitter. For SQLite concurrency tuning, consult the official Python sqlite3 documentation.