Overview

Puppeteer is a Node.js library that provides a high-level API to control headless Chrome or Chromium. When scraping at scale, using residential proxies like Dataimpulse helps avoid IP bans and rate limits. This tutorial shows you how to configure Puppeteer to route traffic through Dataimpulse residential proxies on a Linux system. We'll cover HTTP proxy authentication, testing, and basic rotation.

Prerequisites

Before you begin, ensure you have:

A Linux machine (Ubuntu 20.04+ or similar).
Node.js 14+ and npm installed.
A Dataimpulse account with residential proxy credentials (username, password, endpoint, port). You can sign up for a pay-as-you-go plan on the Dataimpulse website.
Basic familiarity with the command line and JavaScript.

Step 1: Install Node.js and Puppeteer on Linux

First, update your package list and install Node.js if you haven't already.

sudo apt update
sudo apt install -y nodejs npm

Verify the installation:

node --version
npm --version

Create a new project directory and initialize it:

mkdir puppeteer-dataimpulse
cd puppeteer-dataimpulse
npm init -y

Install Puppeteer:

npm install puppeteer

Note: Puppeteer downloads a compatible Chromium binary by default. On Linux, you may need to install additional dependencies. If you encounter missing libraries, install them:

sudo apt install -y libx11-xcb1 libxcomposite1 libxcursor1 libxdamage1 libxi6 libxtst6 libnss3 libcups2 libxss1 libxrandr2 libasound2 libatk1.0-0 libatk-bridge2.0-0 libgtk-3-0

Step 2: Obtain Dataimpulse Proxy Credentials

Proxy host (the endpoint provided by Dataimpulse)
Port (e.g., 823 for HTTP, 824 for SOCKS5)
Username and password for authentication

Dataimpulse uses a single endpoint with rotating IPs by default. For this tutorial, we'll use the HTTP endpoint. Replace the placeholders in the code below with your actual credentials.

Step 3: Configure Puppeteer to Use the Proxy

Create a new file called scrape.js and add the following code to launch Puppeteer with the Dataimpulse proxy.

const puppeteer = require('puppeteer');

(async () => {
  const proxyHost = 'your-proxy-host.dataimpulse.com'; // replace with your host
  const proxyPort = 823; // replace with your port
  const proxyUser = 'your_username';
  const proxyPass = 'your_password';

  const browser = await puppeteer.launch({
    headless: true, // or false for debugging
    args: [
      `--proxy-server=http://${proxyHost}:${proxyPort}`,
      '--no-sandbox',
      '--disable-setuid-sandbox'
    ]
  });

  const page = await browser.newPage();

  // Authenticate with the proxy
  await page.authenticate({
    username: proxyUser,
    password: proxyPass
  });

  // Navigate to a test page to verify the IP
  await page.goto('https://httpbin.org/ip', { waitUntil: 'networkidle2' });

  // Extract the IP address
  const ipInfo = await page.evaluate(() => document.body.innerText);
  console.log('Your IP address:', ipInfo);

  await browser.close();
})();

Run the script:

node scrape.js

If successful, you'll see an IP address different from your local one, belonging to Dataimpulse's pool.

Step 4: Handle Proxy Authentication

Puppeteer's page.authenticate() method works for HTTP proxies. However, if you're using SOCKS5 proxies, Puppeteer does not support SOCKS5 authentication directly. In that case, you might need to use a local proxy converter like gost or proxychains. For this tutorial, we focus on HTTP proxies.

If you prefer to embed credentials in the proxy URL (some setups allow it), you can try:

args: [
  `--proxy-server=http://${proxyUser}:${proxyPass}@${proxyHost}:${proxyPort}`
]

But note that Chrome may not accept credentials in the URL for security reasons. The page.authenticate() approach is more reliable.

Step 5: Test the Proxy Connection

To ensure your requests are going through the proxy and rotating IPs, you can run the script multiple times. Each run should return a different IP address.

for i in {1..3}; do node scrape.js; done

You should see three different IPs. This confirms rotation is working.

Step 6: Implement IP Rotation and Session Control (Optional)

Dataimpulse allows you to control rotation using a session identifier in the username. The exact syntax varies, so refer to your Dataimpulse dashboard or documentation for the correct format. Typically, you can append a session ID to the username to maintain the same IP for multiple requests, or omit it to rotate on every request.

In Puppeteer, you can dynamically set the username per browser instance. For example, if you want a sticky session, you might use:

const sessionId = 'abc123'; // replace with your session ID
const proxyUser = `your_username-session-${sessionId}`;

This lets you maintain a sticky session for a scraping task, which can be useful for sites that track sessions.

Troubleshooting

Authentication errors: Ensure your username and password are correct. If you use special characters, URL-encode them.
Connection timeouts: Check your firewall settings and ensure the proxy port is not blocked. Try a different port if available.
Puppeteer fails to launch: Install missing dependencies. Run ldd chrome | grep not found to find missing libraries (path to Chrome may vary).
Proxy not rotating: Verify you are not using a sticky session username. Each request should get a new IP if no session is specified.
SSL errors: Some sites may reject proxies. Try adding --ignore-certificate-errors to args for testing (not recommended for production).

Summary

In this tutorial, you learned how to integrate Dataimpulse residential proxies with Puppeteer on Linux. You installed Puppeteer, configured proxy authentication, tested IP rotation, and explored session control. This setup enables reliable large-scale web scraping by distributing requests across 90M+ residential IPs. Remember to respect website terms of service and robots.txt. For other scraping tools, the same proxy credentials work with any HTTP client.