Website Technology Lookup Python

Run the hosted Website Technology Lookup Actor on Apify from the web interface without writing code, or integrate it with Python, JavaScript or cURL to look up structured website technology data.

This public repository contains working request examples, sanitized domain input, representative technology results and CSV export code. It is useful both for non-technical users who want to run a domain lookup in Apify and for developers who need an integration example. It documents the integration surface without publishing Actor source code or private runtime configuration.

Browse the technical examples and sample dataset

What this repository helps you do

• Build categorized technology profiles for competitor domains.
• Identify published CMS, hosting, web server, analytics, frontend and security technologies.
• Enrich domain lists for SEO research, sales research, market analysis or lead qualification.
• Keep current detections separate from technologies marked as recently used.
• Process up to 10,000 normalized domains in one hosted Actor run and export the Dataset.

Example result

The repository includes a representative result in data/sample-output.json and a flattened table in data/sample-output.csv.

{
  "domain": "github.com",
  "status": "ok",
  "title": "Github.com",
  "technologies": [
    {
      "category": "Content Management System",
      "name": "GitHub Pages",
      "version": null,
      "detectionStatus": "current",
      "isCurrentlyDetected": true
    }
  ],
  "httpStatusCode": 200,
  "attempts": 1,
  "error": null
}

Run Website Technology Lookup without code

You can look up website technologies directly from the Apify web interface:

1. Open Website Technology Lookup on Apify.
2. In the Input tab, add one or more domains or website URLs to Domains.
3. Review the list and click Start.
4. Open the Dataset tab to inspect one result per normalized domain.
5. Export the results as JSON, CSV, Excel or JSONL for a spreadsheet or another workflow.

For a first test, use github.com or the small payload in data/sample-input.json. The web workflow does not require you to install Python or copy an API token into this repository.

See the complete field-by-field walkthrough in docs/no-code-guide.md.

Try Website Technology Lookup with Apify's free plan

Apify's Free plan includes $5 in monthly prepaid usage that can be spent in the Apify Store or on your own Actors. No credit card is required to start, so a small Website Technology Lookup test can often be run with the monthly credit while it is available.

This is not unlimited free usage. Unused credits expire at the end of the billing cycle and do not roll over. See the current Apify pricing before running a larger domain batch.

Quick start for developers

Python

1. Install the Python client

pip install -r examples/python/requirements.txt

2. Set your Apify token

export APIFY_API_TOKEN="your-token"

On Windows PowerShell:

$env:APIFY_API_TOKEN = "your-token"

3. Run the Python example

python examples/python/website_technology_lookup.py

The example reads data/sample-input.json, starts the hosted Actor and prints the returned Dataset items as JSON lines.

Input example

{
  "domains": [
    "github.com",
    "example.com",
    "https://www.example.org/"
  ]
}

Website URLs are accepted and normalized to their hostname. Equivalent values are deduplicated before processing. See docs/input-reference.md for the complete input contract.

Request examples

cURL

See examples/curl-request.md for the synchronous API request.

Python

See examples/python/website_technology_lookup.py for a complete request and Dataset iteration example.

JavaScript

See examples/javascript/request.mjs for the official JavaScript client.

All examples call the hosted Actor. They do not expose proxy settings, private source code or credentials.

Output fields

Field	Meaning
domain	Normalized hostname submitted to the lookup.
status	ok, not_found, not_crawled or error.
title	Published website title when available, otherwise null.
websiteBackground	Published site background metadata when available, otherwise null.
technologies	Categorized technology records; it can be empty.
httpStatusCode	HTTP response status when one was received.
attempts	Number of lookup attempts used for the domain.
error	Human-readable explanation for a non-ok result, otherwise null.

Each technology record includes category, name, version, description, hints, detectionStatus and isCurrentlyDetected. Read docs/output-reference.md for the complete contract.

Common use cases

Read docs/use-cases.md for workflows covering:

• Competitor technology profiling.
• CMS and hosting research.
• SEO, sales and lead-list enrichment.
• Recurring technology snapshots.

How to look up website technologies with Python

Use examples/python/website_technology_lookup.py to send a domain list to the hosted Actor and iterate over one Dataset item per normalized domain.

For several domain lists, use examples/python/batch_website_technology_lookup.py. It demonstrates a single run with multiple domains.

Website technology lookup API response example

The response is a Dataset of domain-level objects. Use status to distinguish a successful technology lookup from a domain that was not found, has not been crawled or returned an error. Use detectionStatus and isCurrentlyDetected when separating current technologies from recent historical signals.

See data/sample-output.json for the complete representative response fixture.

Export website technology data to CSV

Use examples/python/export_website_technology_csv.py to flatten nested technology records into one CSV row per technology. Domains without technology records remain visible with blank technology columns.

FAQ

See docs/faq.md for answers about URL normalization, duplicate domains, statuses, current versus recently used technologies, batch sizes, scheduling and CSV export.

Limits and pricing

• Up to 10,000 domain values per run.
• Equivalent domains are normalized and processed once.
• One Dataset item is returned for each unique normalized domain outcome.
• One status: "ok" result is one billable event.
• not_found, not_crawled and error results are not charged.
• Current public price: $1 per 1,000 successfully completed domain lookups.

Results are snapshots of publicly available technology information. A later lookup can differ as websites and published data change. The Apify Actor page is authoritative for current pricing and limits.

Hosted version

Use the hosted Actor when you need batching, retries, Dataset storage, API access, recurring schedules or large domain lists without managing the lookup infrastructure yourself:

Run Website Technology Lookup on Apify

Responsible use

Use the returned data lawfully and respect the public source's terms, applicable laws, privacy obligations and third-party rights. The Actor reads publicly available technology information; it does not request a crawl or change the target website. Never commit an Apify token or other credentials to this repository.

Support

For an example or documentation problem, open an issue on GitHub with sanitized input and the error message. For an execution problem, include the Apify run ID but never include your token.

License

This repository is released under the MIT License.