How to set up API data extraction on Semrush

intermediate 12 min read Updated 2026-03-30

Quick Answer

Setting up API data extraction on Semrush requires obtaining API credentials from your account dashboard, configuring authentication with your API key, and making structured HTTP requests to specific endpoints. The process involves generating an API key, understanding rate limits, and formatting requests correctly to extract keyword, domain, or competitor data programmatically.

Full Semrush Review

Prerequisites

Active Semrush subscription with API access
Basic understanding of API concepts and HTTP requests
Programming knowledge or API testing tool like Postman
Valid domain or keywords for data extraction

Step-by-Step Instructions

Access your Semrush API credentials

Log into your Semrush account and navigate to User Settings from the dropdown menu in the top-right corner. Click on API in the left sidebar menu. If you don't see the API option, ensure your subscription plan includes API access. Click Generate New API Key or copy your existing API key. Store this key securely as it will be required for all API requests.

Keep your API key confidential and never share it publicly or commit it to version control systems.

Understand API rate limits and quotas

Review your API usage limits in the same API section of your account settings. Note your daily request quota and rate limits (typically 10 requests per second for most plans). Check the API Units consumption for different endpoints, as some reports consume more units than others. Plan your data extraction schedule to stay within these limits and avoid hitting rate limit errors.

Monitor your API usage regularly to prevent unexpected quota exhaustion during critical data extraction periods.

Choose your API endpoint and parameters

Visit the Semrush API documentation to select the appropriate endpoint for your data needs. Common endpoints include:

/domain_overview for domain metrics
/phrase_organic for organic keyword data
/phrase_adwords for paid search data
/backlinks_overview for backlink metrics

Identify required parameters like domain, phrase, database (country code), and optional parameters like limit and offset for pagination.

Start with simple endpoints like domain_overview to test your setup before moving to more complex data extractions.

Structure your API request

Format your API request URL using this structure:
https://api.semrush.com/?type={report_type}&key={your_api_key}&domain={target_domain}&database={country_code}

For example, to get domain overview data:

https://api.semrush.com/?type=domain_overview&key=YOUR_API_KEY&domain=example.com&database=us

Add additional parameters as needed based on your chosen endpoint and data requirements.

URL-encode special characters in your parameters to avoid request errors.

Set up authentication and headers

Configure your HTTP client or programming language to include proper headers in your requests. Set the User-Agent header to identify your application. If using tools like Postman, create a new request with GET method and paste your formatted URL. For programming implementations, ensure proper error handling for authentication failures and include retry logic for rate limit responses (HTTP 429).

Implement exponential backoff when retrying failed requests to avoid overwhelming the API servers.

Test your API connection

Make your first test request using a simple endpoint like domain_overview with a known domain. Verify you receive a successful HTTP 200 response with data in CSV or JSON format (depending on your format parameter). Check the response headers for remaining quota information. Parse the response to ensure data is formatted as expected and contains the metrics you need.

Use curl or Postman for initial testing before implementing in your preferred programming language.

Implement data parsing and storage

Set up your data processing pipeline to handle the API response format (default is CSV with semicolon separators). Parse the returned data structure and map fields to your database schema or analysis tools. Implement error handling for empty results, API errors, and malformed responses. Set up automated scheduling if you need regular data extraction, ensuring you respect rate limits and quotas.

Cache frequently requested data locally to reduce API calls and improve response times for your applications.

Monitor and optimize your extraction process

Set up logging to track API usage, response times, and errors. Monitor your daily quota consumption in the Semrush dashboard under API settings. Implement alerts for quota thresholds and API errors. Optimize your requests by using appropriate limit parameters and combining related data calls where possible to minimize API unit consumption.

Consider implementing a queue system for large-scale extractions to manage rate limits automatically.

Common Issues & Troubleshooting

Getting 403 Forbidden or authentication errors

Verify your API key is correct and hasn't expired. Check that your subscription plan includes API access. Ensure the API key is properly included in the request URL as the key parameter. Contact Semrush support if your account should have API access but you're still getting authentication errors.

Hitting rate limits with 429 Too Many Requests errors

Implement request throttling to stay under 10 requests per second. Add delays between requests using sleep() functions or rate limiting libraries. Monitor your request frequency and implement exponential backoff when receiving 429 responses. Consider spreading large extractions across multiple days to stay within daily quotas.

Receiving empty results or 'not found' responses

Verify the domain or keyword exists in Semrush's database for your selected country (database parameter). Check that you're using the correct database code (e.g., 'us' for United States, 'uk' for United Kingdom). Some domains may not have sufficient data in certain regions. Try different database parameters or verify the domain has organic/paid search presence.

API responses are incomplete or missing expected data

Check your subscription plan's API access level, as some reports require higher-tier plans. Verify you're using the correct endpoint parameters and haven't exceeded the maximum limit value for the endpoint. Use pagination with offset parameter for large datasets. Ensure the domain has sufficient search visibility for the requested metrics.

Prices mentioned in this guide are pulled from current plan data and may change. Always verify on the official Semrush website before purchasing.

Visit Semrush View Semrush Pricing

Related to Semrush

ReviewSemrush Review PricingSemrush Pricing AlternativesSemrush Alternatives