How to handle API pagination effectively on Make
Quick Answer
Handle API pagination in Make by using Iterator modules with HTTP requests in loops, tracking pagination parameters like page numbers or cursors, and implementing proper termination conditions. Configure bundle settings and use aggregators to collect all paginated results into a single dataset.
Prerequisites
- Basic understanding of Make.com scenarios
- Knowledge of HTTP modules and API endpoints
- Experience with JSON data handling
- Understanding of iteration and loops in Make
Step-by-Step Instructions
1
Set up the initial HTTP request module
Add an HTTP > Make a request module to your scenario. Configure the URL field with your API endpoint and include initial pagination parameters like
?page=1&limit=100. In the Headers section, add your authentication headers. Set the Parse response option to Yes to automatically handle JSON responses. Start with a small limit value (like 10-20 items) during testing to avoid hitting rate limits while debugging your pagination logic.
2
Create variables to track pagination state
Add a Tools > Set variable module before your HTTP request. Create variables for
currentPage (set to 1), hasMoreData (set to true), and allResults (set to empty array []). These variables will track your pagination progress throughout the scenario execution. Use descriptive variable names that clearly indicate their purpose, especially when working with complex pagination scenarios.
3
Configure the repeater module for pagination loop
Add a Flow Control > Repeater module after your variable setup. Set the Initial value to 1 and Repeats to a high number like 1000 (this acts as a safety limit). Connect your HTTP request module inside this repeater loop. Update the HTTP module's URL to use the repeater's current iteration:
{{your-endpoint}}?page={{1.i}}&limit=100.4
Add pagination logic with filters
After your HTTP request, add a Flow Control > Router with two routes. On the first route, add a filter that checks if more data exists:
{{length(2.data)}} > 0 (assuming 'data' is your results array). On the second route, add a filter for {{length(2.data)}} = 0 to handle the end condition. This will control when to continue or stop pagination. Different APIs use different indicators for pagination end - some use 'hasMore' boolean, others use empty arrays or specific response codes.
5
Process and store paginated results
On the first route (where data exists), add an Iterator module to process individual items from
{{2.data}}. Follow this with your data processing modules (filters, transformations, etc.). Use an Array Aggregator module to collect processed results, setting the Source Module to your iterator and Target structure type to Custom. Consider using Tools > Sleep module between API calls if your target API has rate limiting to avoid 429 errors.
6
Handle different pagination styles
For cursor-based pagination, modify your approach to store the next cursor token: use
{{2.next_cursor}} in subsequent requests. For offset-based pagination, calculate offset as {{(1.i - 1) * 100}}. For link-based pagination, extract the next URL from response headers using {{2.headers.Link}} and parse it appropriately. Always check the API documentation to understand which pagination method is used - some APIs support multiple methods.
7
Implement proper error handling and limits
Add an Error Handler route to your HTTP module to catch 429 (rate limit) and 500+ errors. Configure a Tools > Sleep module with exponential backoff (start with 2 seconds, double on each retry). Set a maximum iteration limit in your repeater (like 100) and add a Break module on your empty data route to properly exit the loop.
Log pagination progress using Tools > Set variable to track total items processed - this helps with debugging large datasets.
8
Optimize and finalize the pagination flow
Add a final Array Aggregator after your repeater to combine all paginated results into one bundle. Configure Source Module as your data processor and set appropriate Group by fields if needed. Test with a small dataset first, then gradually increase limits. Monitor your scenario's Operations usage and set appropriate Max number of cycles in scenario settings.
Use scenario scheduling wisely for large pagination jobs - consider breaking them into smaller chunks or running during off-peak hours.
Common Issues & Troubleshooting
Infinite loops consuming all operations
Set a reasonable Repeats limit in your Repeater module and add proper termination conditions. Use Flow Control > Break modules to exit loops when no more data is available.
API rate limiting errors (429 status)
Add Tools > Sleep modules between requests with 1-2 second delays. Implement exponential backoff in error handlers and reduce your request frequency or batch sizes.
Missing data between pagination requests
Check that your pagination parameters are correctly incrementing. Verify the API's pagination method (page-based vs cursor-based) and ensure you're using the correct approach for that specific API.
Memory or timeout issues with large datasets
Process data in smaller chunks by reducing your limit parameter. Use Data Store modules to persist intermediate results and break large pagination jobs into multiple scenario runs.
Prices mentioned in this guide are pulled from current plan data and may change. Always verify on the official Make website before purchasing.