How to Build a Rank Tracker Proxy Rotator with IPRoyal Residential Proxies and Python

Overview

Rank trackers do one thing over and over: query a search engine from a specific location and record where a domain appears. That pattern breaks quickly without proxies. Search engines localize results by the requester's IP, and they throttle or block IPs that query them at volume.

This tutorial shows how to run a Python rank tracker through IPRoyal residential proxies over SOCKS5: build the proxy URL, keep sessions sticky when you need pagination, rotate when you do not, and handle the failures you will actually see in production.

IPRoyal offers residential, ISP, and datacenter proxies over HTTP, SOCKS5, and Shadowsocks-style tunneling, advertises a pool of 32M+ IPs, and bills residential traffic pay-as-you-go with flat monthly plans available on some products. The gateway host, port, username, and password live in your IPRoyal dashboard.

Which proxy type fits rank tracking?

Rotation vs sticky sessions

• Rotate every request when you only need one result page per keyword.
• Hold a sticky session (a few minutes up to roughly 30 minutes) when you follow pagination, expand related-question blocks, or need the same exit IP for a sequence of requests.
• Target the country that matches the locale you report on. A de-DE rank check through a US exit IP is not a German rank check.

Session and country targeting are controlled through the proxy username or a gateway subdomain. Copy the exact pattern from your IPRoyal dashboard rather than guessing; the code below uses the widely seen -session- and -country- placeholders.

Prerequisites

• Python 3.9 or newer
• The SOCKS extras for Requests: pip install "requests[socks]"
• IPRoyal gateway host, port, username, and password from your dashboard
• A keyword file with one row per keyword: query, country code, and locale

Steps

1. Export your credentials so they never live in the source file.

export IPROYAL_HOST="gateway-host-from-dashboard"
export IPROYAL_PORT="gateway-port-from-dashboard"
export IPROYAL_USER="your-username"
export IPROYAL_PASS="your-password"

On Windows PowerShell, use $env:IPROYAL_HOST = "..." instead.

2. Build a proxy URL per request, with optional country and session targeting. Use the socks5h scheme so DNS is resolved at the proxy, not on your machine.

import os
import random
import string
import time

import requests

PROXY_HOST = os.environ['IPROYAL_HOST']
PROXY_PORT = os.environ['IPROYAL_PORT']
PROXY_USER = os.environ['IPROYAL_USER']
PROXY_PASS = os.environ['IPROYAL_PASS']

LOCALE_HEADERS = {
    'en-US': {'Accept-Language': 'en-US,en;q=0.9'},
    'de-DE': {'Accept-Language': 'de-DE,de;q=0.9'},
    'ja-JP': {'Accept-Language': 'ja-JP,ja;q=0.9'},
}


def new_session_id() -> str:
    """Short, unique token used to pin a sticky session."""
    return ''.join(random.choices(string.ascii_lowercase + string.digits, k=8))


def proxy_url(country: str, session_id: str | None = None) -> str:
    user = PROXY_USER
    if session_id:
        user = f'{user}-session-{session_id}'
    if country:
        user = f'{user}-country-{country.lower()}'
    return f'socks5h://{user}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}'

3. Wrap the request in retries with exponential backoff. One failed check should not abort a 2,000-keyword run.

def fetch_serp(query: str, country: str, locale: str,
               session_id: str | None = None, attempts: int = 3) -> str | None:
    proxy = proxy_url(country, session_id)
    proxies = {'http': proxy, 'https': proxy}
    headers = {
        'User-Agent': 'Mozilla/5.0 (compatible; RankTracker/1.0)',
        **LOCALE_HEADERS.get(locale, {}),
    }

    for attempt in range(1, attempts + 1):
        try:
            response = requests.get(
                'https://your-permitted-search-endpoint/search',
                params={'q': query, 'hl': locale},
                proxies=proxies,
                headers=headers,
                timeout=30,
            )
            response.raise_for_status()
            return response.text
        except requests.RequestException as exc:
            print(f'attempt {attempt} failed for {query!r}: {exc}')
            time.sleep(2 ** attempt)

    return None

Point the URL at a search interface you are allowed to query, such as an official API or your own permitted data source. Scraping a search engine may violate its terms of service, so confirm before you scale the workflow.

4. Assign one sticky session per keyword so pagination and follow-up requests reuse the same exit IP.

def check_keyword(row: dict) -> dict:
    session_id = new_session_id()
    html = fetch_serp(row['query'], row['country'], row['locale'], session_id=session_id)
    return {
        'query': row['query'],
        'country': row['country'],
        'position': rank_position(html, row['domain']) if html else None,
        'status': 'ok' if html else 'failed',
    }

5. Extract positions with the parser you already use.

def rank_position(html: str | None, domain: str) -> int | None:
    if not html:
        return None
    # Parse with lxml, selectolax, or BeautifulSoup, match result URLs
    # against `domain`, and return the 1-based index of the first match.
    ...

6. Throttle concurrency. Residential pools are large but not unlimited, and hammering an endpoint from many IPs at once is the fastest route to a block.

from concurrent.futures import ThreadPoolExecutor


def run_batch(keywords: list[dict], workers: int = 8) -> list[dict]:
    with ThreadPoolExecutor(max_workers=workers) as pool:
        return list(pool.map(check_keyword, keywords))

• Start at 5-8 workers and raise the number only while your success rate stays above roughly 95%.
• Add a small random jitter between retries so retries do not arrive in lockstep.

7. Store raw HTML for a short window so you can reparse history without scraping again.

import json
from pathlib import Path


def save_result(result: dict) -> None:
    out = Path('rank-results')
    out.mkdir(exist_ok=True)
    name = result['query'][:40].replace('/', '_')
    with (out / f'{name}.json').open('w', encoding='utf-8') as handle:
        json.dump(result, handle, ensure_ascii=False)

8. Schedule the run and log which proxy configuration produced each result. When a rank changes, you want to know whether the SERP moved or the proxy did.

# Example cron entry: run the tracker at 06:00 and 18:00 local time
0 6,18 * * * cd /srv/rank-tracker && /usr/bin/python3 track.py >> logs/rank.log 2>&1

Troubleshooting

• 407 Proxy Authentication Required: the username or password is wrong, or a special character in the password is not encoded. Percent-encode @, :, /, and # before placing the password in the URL.
• Missing dependencies for SOCKS support: Requests needs the SOCKS extra. Run pip install "requests[socks]" in the same virtual environment as your script.
• Results show the wrong country: country targeting is part of the proxy username. Print proxy_url(...) for one keyword and confirm the expected country code appears before blaming the search engine.
• Every request uses the same IP: you are reusing a session ID. Omit the session component to let the provider rotate.
• A whole country suddenly gets blocked: lower concurrency for that country, rotate more aggressively, and check whether the endpoint has started returning consent or captcha pages instead of results.
• Slower than expected: residential routing adds latency. Raise the timeout to 30-45 seconds for residential or ISP proxies and keep datacenter proxies for non-SERP work.

Summary

Rank tracking succeeds or fails on proxy hygiene. Use IPRoyal residential or ISP proxies with the socks5h scheme for remote DNS, pin a session per keyword when you need continuity, rotate when you do not, and cap concurrency so the pool keeps working. With retries, raw-response storage, and a log of the proxy used, you can tell a real ranking change from a proxy artifact.