How to Scrape Walmart Product Data (2026)

I set out to scrape Walmart product data the quick way: one requests.get at a live product page (/ip/587451676), then a JSON parse to read the price and title. The response came back 200 OK, which looked like a win. The body was about 15 KB, held no __NEXT_DATA__, and contained the string “Robot or human?”. Walmart had served its CAPTCHA page with a success status code, and my parser read nothing.

That failure is the real starting point for how to scrape Walmart product data, because the block is the hard part and the parse is easy once a real page is in hand. Below is exactly what product data a Walmart page exposes, where each field lives, the Python that reads it, and the managed route that returns the same JSON without the block. Every result here comes from tests I ran in July 2026.

What product data can you scrape from a Walmart product page?

You can scrape the product title, price, specifications, availability, seller, and rating from a Walmart product page, because Walmart ships all of it inside one JSON object on every page. Walmart runs on Next.js, so it serializes the full product record into a <script id="__NEXT_DATA__" type="application/json"> tag and leaves the visible HTML as a thin shell. You parse that one blob and read keys instead of writing a selector for every field.

Here is where each field this guide covers actually sits in the JSON:

The full path to the product object is props.pageProps.initialData.data.product. This is the standard Next.js pattern: the framework’s getServerSideProps documentation confirms that server-fetched data is serialized into __NEXT_DATA__ under pageProps so the page can hydrate in the browser. The practical payoff is that price, title, and rating come from one json.loads, not from markup that breaks every time Walmart reshuffles its layout.

The catch is that reaching that JSON is the part Walmart fights. Before the parser, it helps to see what a blocked request actually looks like.

Why does scraping Walmart product data return a CAPTCHA instead of JSON?

Scraping Walmart product data returns a CAPTCHA instead of JSON because Walmart runs PerimeterX (now HUMAN) bot detection, which scores your IP reputation and browser fingerprint, then serves a “Robot or human?” page in place of the product. The detail that trips people up is the status code: Walmart returns that block with HTTP 200, the same code a real page uses, so a naive script treats the CAPTCHA as success and fails later at the parse step with a confusing NoneType error.

I sent the same live product URL three ways from an ordinary datacenter IP in July 2026:

Every variant returned the same ~15 KB shell with “Robot or human?” in it and no product JSON. A real product page is hundreds of kilobytes and carries the __NEXT_DATA__ blob, so a 15 KB body is the interstitial. The User-Agent made no difference, which fits how the vendor works. HUMAN scores three things before you ever see product data:

• IP reputation - datacenter ranges are flagged, residential ones far less so.
• TLS and HTTP fingerprint - the handshake a Python client sends does not match a real browser.
• JavaScript environment - the challenge expects a browser that runs its script and returns sensor data.

Walmart’s robots.txt reinforces this at the policy layer, disallowing /search, /account/, and /api/ while leaving product /ip/ pages crawlable. The fix has to change where the request comes from, not just the header. First the parser, because it does not change once a real page is in hand.

How do you scrape Walmart product data with Python?

You scrape Walmart product data with Python by fetching the page, pulling the __NEXT_DATA__ script tag, and reading the fields from its JSON, using requests and BeautifulSoup. Here is the parser I run once a real product page (not the CAPTCHA page) is in hand:

import json
import requests
from bs4 import BeautifulSoup

UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
      "(KHTML, like Gecko) Chrome/125.0 Safari/537.36")

def scrape_walmart_product(html: str) -> dict:
    soup = BeautifulSoup(html, "html.parser")
    tag = soup.find("script", id="__NEXT_DATA__")
    if tag is None:
        raise RuntimeError("No __NEXT_DATA__ - you got the 'Robot or human?' page")

    data = json.loads(tag.string)
    product = data["props"]["pageProps"]["initialData"]["data"]["product"]
    price = product.get("priceInfo", {}).get("currentPrice", {}) or {}
    reviews = product.get("reviews", {}) or {}

    return {
        "id": product.get("usItemId"),
        "title": product.get("name"),
        "brand": product.get("brand"),
        "price": price.get("price"),
        "currency": price.get("currencyUnit"),
        "availability": product.get("availabilityStatus"),
        "seller": product.get("sellerName"),
        "rating": reviews.get("averageRating", product.get("averageRating")),
        "review_count": reviews.get("numberOfReviews", product.get("numberOfReviews")),
    }

resp = requests.get("https://www.walmart.com/ip/587451676",
                    headers={"User-Agent": UA}, timeout=25)
print(scrape_walmart_product(resp.text))

The RuntimeError guard is the line worth keeping. Without it the script throws a TypeError: 'NoneType' object is not subscriptable deep in the parse, and you waste time blaming your keys when the real problem is that you got the CAPTCHA page. With the guard, the failure tells you what actually happened.

Two field notes from testing. Walmart moves averageRating and numberOfReviews between product and product.reviews depending on the page, so the parser reads product.reviews first and falls back to product. And currentPrice.price is a number like 24.98, while the display string with the currency symbol sits next to it in the same priceInfo block. When I ran this against the live URL from my own machine, resp.text was the “Robot or human?” page and the guard fired exactly as designed, which is the fetch problem the scale section solves. For a full runnable pipeline with CSV output, I keep that in my guide to scraping Walmart with Python. The three fields worth their own section are the ones tutorials tend to drop: specs, seller, and availability.

How do you extract Walmart specs, seller, and availability?

You extract Walmart specs, seller, and availability from the same product JSON: the specification table comes back as name / value rows, the seller as sellerName, and stock as availabilityStatus. These are the fields that carry the real product intelligence, and each has a quirk worth handling.

The specification table is a list of {name, value} pairs, but Walmart nests it differently across categories, so read it defensively and flatten to a plain dict:

def extract_specs(product: dict) -> dict:
    # Walmart parks the spec table under different keys by category,
    # so read defensively and flatten to {name: value}.
    rows = product.get("specifications") or []
    return {r.get("name"): r.get("value") for r in rows if r.get("name")}

If a category returns nothing, open the __NEXT_DATA__ blob in DevTools and search for a spec label you can see on the page, like “Brand” or “Manufacturer”, to find where that category stored the table. The path drift is real, so I never hard-code one deep specifications path across a mixed catalog.

Seller and availability are simpler but carry the nuance most people miss. sellerName tells you who owns the buy box: a value of “Walmart.com” is a first-party listing, and any other name is a third-party marketplace seller, often fulfilled through Walmart Fulfillment Services. That distinction matters more every year, because Walmart Marketplace has grown past 200,000 third-party sellers who now supply the majority of listings on the site, so on a random product URL the seller is more likely to be a third party than Walmart itself. availabilityStatus returns values like IN_STOCK or OUT_OF_STOCK, and it is location-dependent: the same URL can return a different currentPrice and availabilityStatus for a different store or ZIP code, which is the trap that quietly corrupts a price comparison run across one national assumption.

How do you scrape Walmart product data at scale without getting blocked?

You scrape Walmart product data at scale without getting blocked by sending the product URL to a scraper API that rotates residential proxies, clears the PerimeterX check, and returns the parsed JSON, so the block and the proxy pool become the server’s problem. You make one request and get structured fields, with no 200-status CAPTCHA to detect on your end.

ChocoData follows the URL-in, JSON-out shape I used in testing. The call is a single GET against the product endpoint with your Walmart URL and key:

curl "https://chocodata.com/api/v1/walmart/product?url=https://www.walmart.com/ip/587451676&api_key=$CHOCO_API_KEY"

In Python that is one request and a parse, with no browser and no proxy management:

import os
import requests

resp = requests.get(
    "https://chocodata.com/api/v1/walmart/product",
    params={
        "url": "https://www.walmart.com/ip/587451676",
        "api_key": os.environ["CHOCO_API_KEY"],
    },
    timeout=60,
)
product = resp.json()
print(product["title"], product["price"], product["seller"], product["rating"])

For a one-off pull of a few dozen products, the plain parser above is fine once you have a residential proxy in front of it. For continuous collection across thousands of SKUs, offloading the blocking and rotation is usually the cheaper path once you price in your own maintenance time, since the alternative is running a proxy pool, a headless browser farm, and retry logic and keeping all three alive. A ranked side-by-side of the managed Walmart options sits in the best Walmart scrapers roundup. There is also an official door for some of this data, worth knowing before you decide to scrape at all.

Can you get Walmart product data from an official Walmart API?

You can get some Walmart product data from an official Walmart API, but both official programs are gated behind approval and neither is open to someone with just a product URL and a script. The Walmart I/O affiliate API returns catalog and price data to approved affiliates, and it is built for link-based partnerships rather than open extraction. Access requires program approval and an Impact Radius setup, so it is closed to most data teams.

The seller-side Walmart Marketplace APIs expose item, price, and inventory data too, but only for products you sell as an approved Marketplace seller, 1P supplier, or solution provider. Neither program returns the live product page of an item you do not own or represent, and neither exposes the full public review text. That gap between what the official APIs return and what a public product page shows is why most product-data collection comes back to scraping the page itself.

Is it legal to scrape Walmart product data?

Scraping publicly visible Walmart product data sits in a legal gray area in the US, between what the Computer Fraud and Abuse Act covers and what Walmart’s own terms say. The Ninth Circuit in hiQ Labs v. LinkedIn held that scraping data that is public, with no login or password gate, likely does not count as access “without authorization” under the CFAA. Walmart product pages are public, which puts straightforward product-data scraping on the safer side of that line.

The counterweight is Walmart’s contract. Walmart’s Terms of Use prohibit using any automated device to retrieve or scrape site materials without written consent, so public-page scraping can still be a terms-of-service matter even where it is not a CFAA problem. A few lines stay off-limits regardless: anything behind a login, other users’ personal data, and the robots.txt disallowed paths. I go deeper on the case law and policy in my write-up on whether scraping Walmart is legal, and I treat that as the prerequisite before collecting product data at volume.