How to Use ProxyScrape Residential Proxies with Puppeteer for Web Scraping in Node.js
A step-by-step guide to integrating ProxyScrape residential and datacenter proxies with Puppeteer in Node.js. Learn HTTP proxy configuration, authentication, rotation, and a SOCKS5 bridge for advanced scraping.
Overview
ProxyScrape offers residential, datacenter, and mobile proxies with HTTP and SOCKS5 support. When combined with Puppeteer, a headless Chrome browser for Node.js, you can automate web scraping at scale while routing traffic through a pool of 97M+ IPs. This tutorial shows you how to configure ProxyScrape proxies in Puppeteer, authenticate, rotate IPs, and handle common errors.
Prerequisites
- Node.js 16 or later installed. You can download it from the official Node.js website.
- A ProxyScrape account with residential or datacenter proxies. The free trial or paid plan works.
- Basic familiarity with JavaScript and the command line.
Step 1: Create a New Node.js Project
Open your terminal and run:
mkdir puppeteer-proxyscrape
cd puppeteer-proxyscrape
npm init -y
npm install puppeteer
This creates a project folder and installs Puppeteer. Note that Puppeteer downloads a compatible Chromium build automatically.
Step 2: Retrieve Your ProxyScrape Proxy Details
Log in to your ProxyScrape dashboard. Navigate to the proxy list section. You will see connection details for each proxy type:
- Residential proxies: Rotating or sticky sessions, with username/password authentication.
- Datacenter proxies: Static IPs, also with authentication.
Note the following for your chosen proxy:
- Host (e.g.,
gateway.proxyscrape.com— check your dashboard for the exact host) - Port (e.g.,
8000for HTTP,1080for SOCKS5 — again, confirm in dashboard) - Username
- Password
ProxyScrape typically provides HTTP and SOCKS5 endpoints. For this tutorial, we'll start with HTTP.
Step 3: Launch Puppeteer with an HTTP Proxy
Create a file named scrape.js and add the following code. Replace the placeholders with your actual proxy details.
const puppeteer = require('puppeteer');
(async () => {
const proxyHost = 'gateway.proxyscrape.com';
const proxyPort = '8000';
const proxyUser = 'your_username';
const proxyPass = 'your_password';
const browser = await puppeteer.launch({
headless: true,
args: [
`--proxy-server=http://${proxyHost}:${proxyPort}`
]
});
const page = await browser.newPage();
// Authenticate with the proxy
await page.authenticate({
username: proxyUser,
password: proxyPass
});
// Verify the proxy IP
await page.goto('https://httpbin.org/ip', { waitUntil: 'networkidle0' });
const content = await page.content();
console.log(content);
await browser.close();
})();
Run the script:
node scrape.js
You should see the IP address used by the proxy instead of your real IP.
Step 4: Rotate Proxies with a List
ProxyScrape residential proxies often support rotating sessions by using different usernames or session IDs. A common pattern is to cycle through a list of proxy endpoints. Here's an example that rotates between two proxies.
const proxies = [
{ host: 'gateway.proxyscrape.com', port: '8000', user: 'user1', pass: 'pass1' },
{ host: 'gateway.proxyscrape.com', port: '8000', user: 'user2', pass: 'pass2' }
];
(async () => {
for (const proxy of proxies) {
const browser = await puppeteer.launch({
headless: true,
args: [`--proxy-server=http://${proxy.host}:${proxy.port}`]
});
const page = await browser.newPage();
await page.authenticate({ username: proxy.user, password: proxy.pass });
await page.goto('https://httpbin.org/ip');
const ip = await page.evaluate(() => document.body.innerText);
console.log(`Proxy IP: ${ip}`);
await browser.close();
}
})();
For large-scale rotation, consider using a proxy manager or a library like proxy-chain to handle authentication and rotation centrally.
Step 5: Use SOCKS5 Proxies with Puppeteer (Advanced)
Chromium, the browser behind Puppeteer, does not natively support authenticated SOCKS5 proxies. If you must use ProxyScrape's SOCKS5 endpoints, you can run a local HTTP proxy that forwards to SOCKS5. One option is to use the proxy-chain npm package.
Install it:
npm install proxy-chain
Then create a local proxy server:
const ProxyChain = require('proxy-chain');
(async () => {
const socksUrl = 'socks5://username:[email protected]:1080';
const localHttpProxy = await ProxyChain.anonymizeProxy(socksUrl);
console.log(`Local HTTP proxy: ${localHttpProxy}`);
// Pass localHttpProxy as --proxy-server in Puppeteer
const browser = await puppeteer.launch({
headless: true,
args: [`--proxy-server=${localHttpProxy}`]
});
const page = await browser.newPage();
await page.goto('https://httpbin.org/ip');
console.log(await page.content());
await browser.close();
})();
This bridges authenticated SOCKS5 to an unauthenticated local HTTP proxy that Chromium can use.
Troubleshooting
- ERR_PROXY_CONNECTION_FAILED: Check the host and port. Ensure your IP is whitelisted if you use IP-based authentication. ProxyScrape uses username/password by default, but double-check your dashboard.
- Authentication popup / 407 error: Make sure you call
page.authenticate()before navigating. - Slow responses: Try a datacenter proxy for speed or a residential proxy for stealth. Adjust timeouts in Puppeteer.
- SOCKS5 not working: Remember that Chromium doesn't support SOCKS5 auth directly. Use the
proxy-chainworkaround above.
Summary
You have learned how to configure ProxyScrape residential and datacenter proxies with Puppeteer in Node.js. We covered HTTP proxy setup, authentication, rotation, and a SOCKS5 bridge for advanced use cases. For reliable infrastructure, ProxyScrape's 97M+ IP pool and stable uptime make it a solid choice for web scraping projects.