Automatic Chunking for Large Files/Prompts to Prevent Anthropic 429 Errors

[Captainhum]

Feature Request: Automatic Chunking/Batching for Anthropic API Usage

Context

Many users working with Kilo Code and Anthropic's Claude API are regularly hitting strict rate limits (429 errors), especially when working with large files, diffs, or batch prompts. The current workflow requires users to manually split files/prompts or use external scripts to avoid exceeding the per-request and per-minute token quotas imposed by Anthropic (e.g., 20,000 tokens/minute per org, max 2,000 tokens/request recommended for safety).

Problem

• Kilo Code does not natively provide an option to automatically split ("chunk") large files or prompts into smaller, API-safe requests.
• Users are forced to manually preprocess their data or write custom scripts, which breaks the seamless experience and is a major source of frustration.
• The lack of this feature is a blocker for advanced/enterprise use cases (batch refactoring, large codebase analysis, etc.) and leads to repeated 429 errors and wasted API quota.

Proposed Solution

• Add an option in Kilo Code to automatically split large files/prompts into smaller chunks that respect Anthropic’s rate and size limits.
• Allow users to configure chunk size (tokens/chars/lines), batch size, and delay between batches.
• Optionally, provide feedback in the UI/logs about how data is being chunked and how many API calls will be made.
• Implement exponential backoff/retry for 429 errors.

Benefits

• Prevents 429 errors and improves user experience.
• Makes Kilo Code usable for large-scale and batch operations without manual intervention.
• Aligns with best practices discussed in the Anthropic and Kilo Code communities (GitHub, Reddit, etc.).

References

• Community discussions on GitHub and Reddit about this pain point.
• Anthropic API documentation on rate limits: https://docs.anthropic.com/en/api/rate-limits

Thank you for considering this feature! This would make Kilo Code much more robust and user-friendly for all users working with Anthropic.

This request was generated on behalf of a user experiencing repeated 429 errors despite following all best practices. If more info or user stories are needed, please reply here.