# Apify Documentation > Apify is the largest marketplace of tools for AI. Thousands of ready-made Actors to automate your business. Get real-time web data, track competitors, generate leads, analyze sentiment, and orchestrate your apps. Actors are created by a global community of builders earning over $1M every month. Apify takes care of infrastructure, billing, and distribution. The entire content of Apify documentation is available in a single Markdown file at https://docs.apify.com/llms-full.txt For pricing details, see https://apify.com/pricing.md For autonomous AI agents, the Apify Agent General Interface (AGI) at https://agi.apify.com is the entry point to use and pay for the Apify platform and Actors through agentic payment protocols like x402 or MPP, without account setup or ongoing billing. ## Apify API - [Apify API documentation](https://docs.apify.com/api/llms.txt): Endpoints to manage Actors, runs, builds, tasks, datasets, key-value stores, request queues, schedules, webhooks, and your account. ## Apify API client for JavaScript - [Overview](https://docs.apify.com/api/client/js/docs.md): The official JavaScript library to access the Apify API, with automatic retries, TypeScript support, and cross-platform compatibility. - [Quick start](https://docs.apify.com/api/client/js/docs/introduction/quick-start.md): Get started with the Apify API client for JavaScript by running an Actor and retrieving results from its dataset. - [Usage patterns](https://docs.apify.com/api/client/js/docs/concepts/usage-patterns.md): Learn the resource client and collection client patterns used by the Apify API client for JavaScript. - [Convenience functions](https://docs.apify.com/api/client/js/docs/concepts/convenience-functions.md): Use call(), waitForFinish(), and other convenience functions in the Apify API client for JavaScript. - [Error handling and retries](https://docs.apify.com/api/client/js/docs/concepts/error-handling.md): Handle API errors and configure automatic retries with exponential backoff in the Apify API client for JavaScript. - [Pagination](https://docs.apify.com/api/client/js/docs/concepts/pagination.md): Paginate through large result sets and use async iteration with the Apify API client for JavaScript. - [Use in bundled environments](https://docs.apify.com/api/client/js/docs/concepts/bundled-environments.md): Use the Apify API client for JavaScript in browsers, Cloudflare Workers, and other edge runtimes. - [Code examples](https://docs.apify.com/api/client/js/docs/guides/examples.md): Practical code examples for common tasks with the Apify API client for JavaScript, including webhooks and datasets. - [Changelog](https://docs.apify.com/api/client/js/docs/changelog.md) - [apify-client reference](https://docs.apify.com/api/client/js/reference.md) ## Apify API client for Python - [Overview](https://docs.apify.com/api/client/python/docs.md): The official Python library to access the Apify API, with automatic retries, async support, and comprehensive API coverage. - [Quick start](https://docs.apify.com/api/client/python/docs/introduction/quick-start.md): Get started with the Apify API client for Python by running an Actor and retrieving results from its dataset. - [Single and collection clients](https://docs.apify.com/api/client/python/docs/concepts/single-and-collection-clients.md): Understand the two main types of clients for managing API resources. - [Convenience methods](https://docs.apify.com/api/client/python/docs/concepts/convenience-methods.md): Convenience methods to handle actions that the API alone cannot perform efficiently. - [Error handling](https://docs.apify.com/api/client/python/docs/concepts/error-handling.md): The client automatically extracts data, converts date strings to datetime objects, and raises ApifyApiError on errors. - [Pagination](https://docs.apify.com/api/client/python/docs/concepts/pagination.md): Work with paginated data using the ListPage object returned by list methods. - [Asyncio support](https://docs.apify.com/api/client/python/docs/concepts/asyncio-support.md): Use the asynchronous ApifyClientAsync to interact with the Apify API using Python's async/await syntax. - [Retries](https://docs.apify.com/api/client/python/docs/concepts/retries.md): Automatic retries for requests that fail due to network issues or rate limiting. - [Nested clients](https://docs.apify.com/api/client/python/docs/concepts/nested-clients.md): Use nested clients to simplify working with related collections like Actor runs. - [Streaming resources](https://docs.apify.com/api/client/python/docs/concepts/streaming-resources.md): Stream dataset items, key-value store records, and logs incrementally without downloading entirely into memory. - [Logging](https://docs.apify.com/api/client/python/docs/concepts/logging.md): Configure the apify_client logger to print debug information about API requests. - [Passing input to Actor](https://docs.apify.com/api/client/python/docs/guides/passing-input-to-actor.md): Run an Actor and retrieve results by passing input data directly to the call method. - [Retrieve Actor data](https://docs.apify.com/api/client/python/docs/guides/retrieve-actor-data.md): Retrieve, paginate, merge, and export Actor output data from datasets. - [Manage tasks for reusable input](https://docs.apify.com/api/client/python/docs/guides/manage-tasks-for-reusable-input.md): Create multiple tasks with different input configurations for the same Actor. - [Integration with data libraries](https://docs.apify.com/api/client/python/docs/guides/integration-with-data-libraries.md): Load dataset items directly into a Pandas DataFrame for efficient manipulation and analysis. - [Upgrading to v2](https://docs.apify.com/api/client/python/docs/upgrading/upgrading-to-v2.md): Breaking changes between Apify Python API Client v1.x and v2.0. - [Changelog](https://docs.apify.com/api/client/python/docs/changelog.md) - [apify-client reference](https://docs.apify.com/api/client/python/reference.md) ## Experimental API clients - [Apify API client for Rust](https://raw.githubusercontent.com/apify/apify-client-rust/refs/heads/master/README.md). Install by adding `apify-client` to `Cargo.toml`. - [Apify API client for Go](https://raw.githubusercontent.com/apify/apify-client-go/refs/heads/master/README.md). Install with `go get github.com/apify/apify-client-go`. - [Apify API client for PHP](https://raw.githubusercontent.com/apify/apify-client-php/refs/heads/master/README.md). Install with `composer require apify/apify-client`. - [Apify API client for Java](https://raw.githubusercontent.com/apify/apify-client-java/refs/heads/master/README.md). Install from Maven Central as `com.apify:apify-client`. - [Apify API client for .NET](https://raw.githubusercontent.com/apify/apify-client-dotnet/refs/heads/master/README.md). Install with `dotnet add package Apify.Client`. ## Apify SDK for JavaScript - [Apify SDK for JavaScript](https://docs.apify.com/sdk/js/docs/overview.md): The official library for creating Apify Actors in JavaScript, providing tools for web crawling, scraping, and automation at scale. - [Quick start](https://docs.apify.com/sdk/js/docs/introduction/quick-start.md): Get started with the Apify SDK for JavaScript by creating your first Actor. - [Actor lifecycle](https://docs.apify.com/sdk/js/docs/concepts/actor-lifecycle.md): Understanding Actor lifecycle, initialization, and the Apify platform. - [Environment Variables](https://docs.apify.com/sdk/js/docs/concepts/environment-variables.md): Environment variables used by Apify SDK. - [Proxy Management](https://docs.apify.com/sdk/js/docs/concepts/proxy-management.md): IP address rotation and proxy configuration. - [Request Storage](https://docs.apify.com/sdk/js/docs/concepts/request-storage.md): Request storage types for managing crawl queues. - [Result Storage](https://docs.apify.com/sdk/js/docs/concepts/result-storage.md): Result storage types for saving crawl output. - [Session Management](https://docs.apify.com/sdk/js/docs/concepts/session-management.md): SessionPool for managing browser sessions and cookies. - [Running in Docker](https://docs.apify.com/sdk/js/docs/concepts/docker-images.md): Example Docker images to run your crawlers. - [Pay-per-event Monetization](https://docs.apify.com/sdk/js/docs/concepts/pay-per-event.md): Monetize your Actors using the pay-per-event pricing model. - [Basic crawler](https://docs.apify.com/sdk/js/docs/guides/basic-crawler.md): Bare-bones example using BasicCrawler. - [Cheerio crawler](https://docs.apify.com/sdk/js/docs/guides/cheerio-crawler.md): Crawl URLs using plain HTTP requests and parse HTML with Cheerio. - [Playwright crawler](https://docs.apify.com/sdk/js/docs/guides/playwright-crawler.md): Use PlaywrightCrawler for browser-based crawling. - [Puppeteer crawler](https://docs.apify.com/sdk/js/docs/guides/puppeteer-crawler.md): Use PuppeteerCrawler for browser-based crawling. - [Crawl all links on a website](https://docs.apify.com/sdk/js/docs/guides/crawl-all-links.md): Use enqueueLinks() to crawl entire websites. - [Crawl a sitemap](https://docs.apify.com/sdk/js/docs/guides/crawl-sitemap.md): Download and crawl URLs from a sitemap. - [Accept user input](https://docs.apify.com/sdk/js/docs/guides/accept-user-input.md): Accept and log user input in your Actor. - [Add data to dataset](https://docs.apify.com/sdk/js/docs/guides/add-data-to-dataset.md): Save data to the default dataset. - [Call actor](https://docs.apify.com/sdk/js/docs/guides/call-actor.md): Start an Apify Actor programmatically. - [Setting up a TypeScript project](https://docs.apify.com/sdk/js/docs/guides/typescript-setup.md): TypeScript support in Apify SDK. - [Upgrading to v3](https://docs.apify.com/sdk/js/docs/upgrading/upgrading-to-v3.md): Breaking changes between Crawlee (v3) and Apify SDK (v2). - [Changelog](https://docs.apify.com/sdk/js/docs/changelog.md) - [apify reference](https://docs.apify.com/sdk/js/reference.md) ## Apify SDK for Python - [Overview](https://docs.apify.com/sdk/python/docs/overview.md): The official library for creating Apify Actors in Python, providing tools for web scraping, automation, and data storage integration. - [Quick start](https://docs.apify.com/sdk/python/docs/quick-start.md): Get started with the Apify SDK for Python by creating your first Actor. - [Actor lifecycle](https://docs.apify.com/sdk/python/docs/concepts/actor-lifecycle.md): How an Apify Actor starts, runs, and shuts down. - [Actor input](https://docs.apify.com/sdk/python/docs/concepts/actor-input.md): Read Actor input from the default key-value store. - [Actor configuration](https://docs.apify.com/sdk/python/docs/concepts/actor-configuration.md): Configure the Actor class using the Configuration class and environment variables. - [Actor events & state persistence](https://docs.apify.com/sdk/python/docs/concepts/actor-events.md): Handle Actor events sent by the Apify platform during runtime. - [Working with storages](https://docs.apify.com/sdk/python/docs/concepts/storages.md): Work with default or named storages for datasets, key-value stores, and request queues. - [Proxy management](https://docs.apify.com/sdk/python/docs/concepts/proxy-management.md): Work around IP blocking using proxy servers. - [Interacting with other Actors](https://docs.apify.com/sdk/python/docs/concepts/interacting-with-other-actors.md): Methods to interact with other Actors and tasks on the platform. - [Accessing Apify API](https://docs.apify.com/sdk/python/docs/concepts/access-apify-api.md): Access Apify API features not covered by the SDK. - [Creating webhooks](https://docs.apify.com/sdk/python/docs/concepts/webhooks.md): Configure webhooks to trigger actions on Actor events. - [Logging](https://docs.apify.com/sdk/python/docs/concepts/logging.md): Logging through Python's standard logging module. - [Pay-per-event monetization](https://docs.apify.com/sdk/python/docs/concepts/pay-per-event.md): Monetize your Actors using the pay-per-event pricing model. - [Using Crawlee](https://docs.apify.com/sdk/python/docs/guides/crawlee.md): Use the Crawlee library in your Actors. - [Using BeautifulSoup with HTTPX](https://docs.apify.com/sdk/python/docs/guides/beautifulsoup-httpx.md): Use BeautifulSoup with HTTPX in your Actors. - [Using Playwright](https://docs.apify.com/sdk/python/docs/guides/playwright.md): Use Playwright for web scraping in your Actors. - [Using Scrapy](https://docs.apify.com/sdk/python/docs/guides/scrapy.md): Use the Scrapy framework in your Actors. - [Using Selenium](https://docs.apify.com/sdk/python/docs/guides/selenium.md): Use Selenium for web scraping in your Actors. - [Using Parsel with Impit](https://docs.apify.com/sdk/python/docs/guides/parsel-impit.md): Combine Parsel and Impit libraries for scraping. - [Running webserver](https://docs.apify.com/sdk/python/docs/guides/running-webserver.md): Run a web server inside your Actor. - [Upgrading to v3](https://docs.apify.com/sdk/python/docs/upgrading/upgrading-to-v3.md): Breaking changes between Apify Python SDK v2.x and v3.0. - [Changelog](https://docs.apify.com/sdk/python/docs/changelog.md) - [apify-sdk-python reference](https://docs.apify.com/sdk/python/reference.md) ## Apify CLI - [Apify CLI overview](https://docs.apify.com/cli/docs.md): An introduction to Apify CLI, a command-line interface for creating, developing, building, and running Apify Actors and managing the Apify cloud platform. - [Quick start](https://docs.apify.com/cli/docs/quick-start.md): Learn how to create, run, and manage Actors using Apify CLI. - [Installation](https://docs.apify.com/cli/docs/installation.md): Learn how to install Apify CLI using installation scripts, Homebrew, or NPM. - [Command reference](https://docs.apify.com/cli/docs/reference.md): The Apify CLI provides tools for managing your Apify projects and resources from the command line. - [Integrating Scrapy projects](https://docs.apify.com/cli/docs/integrating-scrapy.md): Learn how to run Scrapy projects as Apify Actors and deploy them on the Apify platform. - [Telemetry](https://docs.apify.com/cli/docs/telemetry.md): Apify collects telemetry data about the general usage of the CLI. - [Changelog](https://docs.apify.com/cli/docs/changelog.md) ## Apify Academy - [Apify Academy](https://docs.apify.com/academy/llms.txt): Free courses and tutorials on web scraping in JavaScript and Python, browser automation, anti-scraping protections, and building and deploying Actors. ## Legal documents - [Apify Legal](https://docs.apify.com/legal/llms.txt): Terms and conditions, privacy and cookie policies, data processing addendums, and the terms for publishing Actors in Apify Store. ## account - [Account](https://docs.apify.com/account.md): Manage your Apify account, billing, security, organization, and access permissions. - [Apify AI](https://docs.apify.com/account/apify-ai.md): Apify AI is the conversational AI interface inside Apify Console for finding and running Actors using natural language, instead of keyword search. - [Billing](https://docs.apify.com/account/billing.md): View invoices, track current billing cycle usage, manage subscriptions and limits, and review historical usage from the Billings page in Console. - [Apply a promo code](https://docs.apify.com/account/billing/promo-codes.md): Learn how to apply a promo code to your invoice. - [Collaboration](https://docs.apify.com/account/collaboration.md): Learn how to collaborate with other users and manage permissions for organizations or private resources such as Actors, Actor runs, and storages. - [Grant access rights](https://docs.apify.com/account/collaboration/access-rights.md): Manage permissions for your private resources such as Actors, Actor runs, and storages. Allow other users to read, run, modify, or build new versions. - [General resource access](https://docs.apify.com/account/collaboration/general-resource-access.md): Control how Apify resources are shared. Set default access to open ID-based access or restricted, and manage link sharing and pre-signed URLs. - [List of permissions](https://docs.apify.com/account/collaboration/list-of-permissions.md): Learn about the access rights you can grant to other users. See all access options for Apify resources such as Actors, tasks, storage, and schedules. - [Organization account](https://docs.apify.com/account/collaboration/organization.md): Create a specialized account for your organization to encourage collaboration and manage permissions. Convert an existing account, or create one from scratch. - [Using the organization account](https://docs.apify.com/account/collaboration/organization/how-to-use.md): Learn to use and manage your organization account using Apify Console or the Apify API. View the organizations you are in and manage your memberships. - [Setup](https://docs.apify.com/account/collaboration/organization/setup.md): Configure your organization account by inviting new members and assigning their roles. Manage team members' access permissions to the organization's resources. - [Apify Console](https://docs.apify.com/account/console.md): Get started with Apify Console: create an account, sign in with email, Google, or GitHub, and manage your web scraping projects from one place. - [Limits](https://docs.apify.com/account/limits.md): View default resource limits for the Apify platform, including max memory, disk size, and number of Actors and tasks per user per plan tier. - [Account settings](https://docs.apify.com/account/settings.md): Learn how to manage your Apify account, configure integrations, create and manage organizations, and set notification preferences in the Settings tab. - [Apify subscription changes](https://docs.apify.com/account/subscription-changes.md): Learn what happens to the price and prepaid usage when you upgrade or downgrade your subscription plan or switch from monthly to annual billing. - [Apify subscription plans](https://docs.apify.com/account/subscriptions.md): Learn which Apify plan best fits your needs, how to subscribe, and how usage limits and overage charges work when you exceed your prepaid usage. - [Apply VAT exemption](https://docs.apify.com/account/subscriptions/vat-exemption.md): Learn how to get VAT-exempt invoices for your Apify subscription and how to format your VAT ID. Check what to do if your VAT ID is rejected. - [Two-factor authentication](https://docs.apify.com/account/two-factor-authentication.md): Enable two-factor authentication on your Apify Console account using an authenticator app to add a second layer of security against unauthorized access. ## actors - [Actors](https://docs.apify.com/actors/llms.txt): Develop, test, and deploy Actors, run them with tasks, schedules, and monitoring, then publish and monetize them in Apify Store. ## agents - [Connect your AI agent to Apify](https://docs.apify.com/agents.md): Connect an AI agent to Apify through the MCP server, a plugin, or Agent Skills, then pick the setup guide for Claude Code, Codex, Cursor, and more. - [Let your AI agent pay for Actor runs](https://docs.apify.com/agents/payments.md): Let an AI agent pay for Apify Actor runs without signing up, by buying a prepaid, spend-capped API token from Apify AGI over x402, MPP, or Skyfire. - [Install the Apify plugin in your AI agent](https://docs.apify.com/agents/plugins.md): Install the Apify plugin to give your coding agent the MCP server, Agent Skills, and a routing agent in one step. Pick your agent to get its commands. - [Agent quickstart](https://docs.apify.com/agents/quickstart.md): Connect your AI agent to the Apify MCP server, run your first Actor from a single prompt, and cap what a run your agent starts can cost you. - [Resources for AI agents](https://docs.apify.com/agents/resources.md): Read Apify documentation programmatically through Markdown endpoints, llms.txt indexes, and MCP tools, and find every agent-facing file Apify publishes. - [Apify Agent Skills](https://docs.apify.com/agents/skills.md): Install Apify Agent Skills to give a coding agent tested workflows for scraping, Actor development, actorization, and building Apify integrations. ## console - [Apify Store](https://docs.apify.com/console/store.md): Browse thousands of pre-built Actors in Apify Store, filter by category or pricing, and run them immediately or save settings for later use. ## get-started - [What is Apify](https://docs.apify.com/get-started.md): Learn what Apify is, how Actors turn input into results on the platform, how its capabilities fit together, and where to start for your goal. - [Build Actors](https://docs.apify.com/get-started/build-actors.md): Build your own Actor on the Apify platform - create it from a template, understand the Actor model, deploy it, publish it to Apify Store, and set a price. - [Integrate Actors](https://docs.apify.com/get-started/integrate.md): Connect Apify to your stack - call Actors from your code, react to runs with webhooks, and plug results into your workflow tools, data stores, and agents. - [Run Actors](https://docs.apify.com/get-started/run-actors.md): Run existing Actors from Apify Store to get data or automate tasks with no code - pick an Actor, configure input, collect results, then automate. ## Glossary - [Glossary](https://docs.apify.com/glossary.md): Definitions of Apify-specific terms, from Actors and datasets to proxy types and pricing models, each linking to the page that covers it in full. ## integrations - [Integrations](https://docs.apify.com/integrations/llms.txt): Connect Actors to workflow tools like n8n and Zapier, AI agents and frameworks, data warehouses, and your own code through the Apify API and webhooks. ## mcp - [Apify MCP server](https://docs.apify.com/mcp.md): Learn how to use the Apify MCP server to integrate Apify's library of Actors into your AI agents or large language model-based applications. ## proxy - [Proxy](https://docs.apify.com/proxy.md): Learn to anonymously access websites in scraping/automation jobs. Improve data outputs and efficiency of bots, and access websites from various geographies. - [Datacenter proxy](https://docs.apify.com/proxy/datacenter-proxy.md): Learn how to reduce blocking when web scraping using IP address rotation. See proxy parameters and learn to implement Apify Proxy in an application. - [Estimate how many datacenter IP addresses you need](https://docs.apify.com/proxy/estimate-ip-addresses.md): Estimate how many datacenter proxy IP addresses a job needs, compare the result with your plan, and get more with an add-on or a plan upgrade. - [Google SERP proxy](https://docs.apify.com/proxy/google-serp-proxy.md): Learn how to collect search results from Google Search-powered tools. Get search results from localized domains in multiple countries, e.g. the US and Germany. - [Residential proxy](https://docs.apify.com/proxy/residential-proxy.md): Achieve a higher level of anonymity using IP addresses from human users. Access a wider pool of proxies and reduce blocking by websites' anti-scraping measures. - [Unblocker](https://docs.apify.com/proxy/unblocker.md): Bypass anti-bot and anti-captcha systems automatically when scraping, without building or maintaining your own bypass logic for blocked websites. - [Use your own proxies](https://docs.apify.com/proxy/using-your-own-proxies.md): Add your own proxy URLs to Actor runs in Apify Console or configure them in the Apify SDK using the proxy configuration API for JavaScript and Python. ## security - [Security](https://docs.apify.com/security.md): Apify's security practices, compliance certifications, and vulnerability disclosure policy for protecting your Actors, data, and the platform. - [Platform architecture](https://docs.apify.com/security/architecture.md): How the Apify platform is built - control and execution planes, what happens during an Actor run, how workloads stay isolated, and how storage works. - [Shared responsibility model](https://docs.apify.com/security/shared-responsibility.md): How security responsibilities are divided between Apify and customers across the platform, your account, your Actors, and the data you collect. - [Vulnerability disclosure policy](https://docs.apify.com/security/vulnerability-disclosure.md): Apify's vulnerability disclosure policy for the platform and open-source projects, including scope, testing guidelines, safe harbor, and how to report one. ## storage - [Storage](https://docs.apify.com/storage.md): Store anything from images and key-value pairs to structured output data. Learn how to access and manage your stored data from the Apify platform or via API. - [Dataset](https://docs.apify.com/storage/dataset.md): Store and export web scraping, crawling or data processing job results. Learn how to access and manage datasets in Apify Console or via API. - [Dataset schema specification](https://docs.apify.com/storage/dataset-schema.md): Define a dataset schema to control how your Actor's results are structured, validated, and displayed in Apify Console, the API, and integrations. - [Use multiple datasets](https://docs.apify.com/storage/dataset-schema/multiple-datasets.md): Define more than one dataset in your Actor, write to each from your code, and surface them on the run page so users can find the right output. - [Validate dataset output](https://docs.apify.com/storage/dataset-schema/validation.md): Specify the dataset schema within your Actors to add monitoring and validation at the field level, ensuring data quality and consistency across runs. - [Key-value store](https://docs.apify.com/storage/key-value-store.md): Store anything from Actor or task run results, JSON documents, or images. Learn how to access and manage key-value stores from Apify Console or via API. - [Key-value store schema specification](https://docs.apify.com/storage/key-value-store-schema.md): Define a key-value store schema to organize records into collections with content type validation, making your Actor's key-value store easier to navigate. - [Request queue](https://docs.apify.com/storage/request-queue.md): Queue URLs for an Actor to visit in its run. Add requests in batches, lock them across runs, and manage request queues from Apify Console or via API. - [Share storage](https://docs.apify.com/storage/share.md): Grant access rights to your storages, share them by link using their ID or name, and generate time-limited pre-signed URLs for restricted resources. - [Use storage from another run](https://docs.apify.com/storage/use-from-another-run.md): Open a dataset, key-value store, or request queue that belongs to another Actor or task run by its name or ID, from the SDK, API clients, or the API.