How to Scrape Zalando and ASOS: European Fashion Retail Data

Key Takeaways:

• Running a Zalando and ASOS scraper requires handling an advanced anti-bot system and handling dynamic page content.
• Best practices include ensuring session consistency, reverse-engineering internal and mobile APIs, and selecting high-quality proxies.
• With complex targets like retail websites, hiring a professional scraping company can help get results faster.

Your ASOS scraper may work one day and return a 403 error the next. Big fashion retailers invest heavily in anti-bot protections and update them on a regular basis. If you’re building a data-driven product or analytics system, prolonged pipeline downtime can hinder the accuracy of your insights.

In this article, we explain the challenges your retail scraper faces and show how to overcome them when running large-scale collection.

The Main Challenges of Running a Zalando Scraper

Automated data collection from platforms like ASOS and Zalando is often difficult to maintain. Below is a list of the most common issues to help you assess the risks before you start scraping and troubleshoot when something goes wrong.

1. Legal challenges of scraping in Europe

Although collecting publicly available data is generally legal across the world, the particular laws differ depending on use cases, data types, and collection methods. While non-EU European countries often have less strict approaches, the European Union is one of the most regulated markets for web scraping.

Read our article to find out more about the legality of web scraping.

Before building a Zalando scraper that covers Germany and other EU states, we recommend consulting a lawyer or doing your own research. Here’s a brief overview of regulations to consider.

1. Personal data. Names, face photos, and phone numbers are protected by the GDPR. Collecting them requires explicit consent or a documented legitimate interest. When scraping reviews, make sure you do not gather or store any personal or identifiable reviewer details.
2. Sources protected by a website’s Terms of Service. The EU courts view ToS as a form of a legally binding contract. In Ryanair v. PR Aviation, the price comparison platform was ruled to stop scraping Ryanair’s website in violation of its ToS. Note that both ASOS and Zalando prohibit “screen scraping” in their Terms & Conditions.

2. Akamai Bot Manager

If you’re clear legally, the next obstacle is websites’ anti-bot protections. Both ASOS and Zalando use Akamai Bot Manager — a system that passes each request through an edge server network where detection happens. It assesses traffic using a set of parameters:

• IP and ASN reputation. Akamai owns a dedicated database and records the history of each address’s activities across all websites that use Akamai’s services.
• TLS and HTTP fingerprints. All the session’s signals need to be consistent.
• Device and browser telemetry. The system uses JavaScript Challenges to assess additional signals.
• Behavioral signals. Akamai uses AI to analyze mouse movements and typing patterns and flag artificial sessions.

The system is maintained and regularly updated. Even if you’ve managed to bypass it once, it doesn’t guarantee continuous uptime.

3. Dynamic content

Zalando uses client-side JavaScript rendering for all the product data. That means a page’s HTML is empty before a browser loads the dynamic content. If your scraper relies on HTTP requests, it won’t return any data.

For ASOS, dynamic rendering is partial. As a result, a BeautifulSoup-based scraper can appear functioning until you try to use the collected data. You can get the product title and image but miss the price, as happened in this case. For reliable results, use JS rendering or pull data from another source.

Best Practices for Building an ASOS Scraper

Following the right practices from the start can help keep your data pipeline running as your needs grow. Below are a few essential steps to follow, along with additional alternative ways to get fashion data at scale.

1. Select the right proxies

With advanced protection systems like Akamai, cheap datacenter proxies are blocked before a scraper can load a single page. The same can be true for residential or mobile IPs that have already been used to scrape Akamai-protected websites without taking the right precautions.

Choose a reputable proxy provider, read reviews, and try the addresses on a small run before committing.

2. Manage geo-specific signals

Zalando has separate versions for different European countries. When accessing each one, make sure your IP and browser signals match the particular location. Foreign proxies are not blocked automatically, but inconsistent geographic and language markers within a single session can trigger a CAPTCHA or make the system watch you more closely.

3. Reverse engineer internal APIs

Fashion retailers host millions of items. If your analysis needs to cover a large portion of the data, scraping it page-by-page will take a lot of time and resources. In addition, large scale makes it harder to imitate human sessions. If you send too many requests without adding natural pauses and rotating user agents, you’ll get blocked. Finding an API is a popular alternative.

Retail websites don’t have official public APIs third-party developers could use, but they still use API connections for internal data transfer. Finding the endpoints can help pull data without crawling. Here’s a quick guide:

1. Open your browser’s Developer Tools on a page containing the data you need.
2. Inspect network requests and identify the relevant one.
3. Copy it as cURL to reproduce the request.
4. Test it outside the browser, for example, in a terminal.
5. Determine which parameters are required for the request to work consistently by removing them one by one.
6. Recreate the request in your scraper.

This is not a universal solution, as the internal API are also often protected by Akamai or other tools. In addition, they can change at any moment, requiring continuous monitoring and maintenance.

4. Try scraping mobile apps

When building an ASOS scraper, app version of the platform can offer another internal API to work with. A reverse-engineered mobile API can deliver clean, structured JSON that is easier to integrate than raw scraped HTML. As protections evolve, one target can become easier than another. Run tests before investing in a full-scale pipeline.

5. Consider professional services

In-house development doesn’t make your ASOS scraper free. Even if you have a team necessary to build a reliable pipeline, the costs add up. The scraping market offers ready-to-use solutions as well as custom services that can deliver your scraper fast. The best option for you depends on your use case.

You can find a detailed cost comparison across options in our article Build vs Buy Data Pipeline.

Read more about scraping retail websites with DataOx.

Before buying an ASOS scraper, review the integration and scalability options. We recommend taking a long-term approach. Pre-built tools work for one-time competitor assessment or small scheduled scraping. If the data needs to be part of your workflow or product, custom development can offer more.

DataOx — Your Trusted Partner for Monitoring Fashion Retail

DataOx builds custom data collection pipelines that go beyond scraping. We handle complex websites, process the extracted data, and tailor delivery to clients’ needs. We can build the data into a bespoke web-based ASOS scraper app, a dashboard, or a custom bot.

After delivery, we can stay on to maintain the pipeline or hand you full documentation for in-house work. The IP rights to the developed solution belong to the client. You get tailored web scraping services without the risk of vendor lock-in.