Isoxya
Scalable web crawling and scraping.
Available on-premises or in the cloud.
Available on-premises or in the cloud.
Isoxya is an extensible data processing system for crawling and scraping the internet. It is written in a compiled, statically typed language for speed and reliability. Isoxya is free, open-source, and packaged as a container. The API can be used by CLI scripts or called by another project. Various plugins are available, many also open-source.
Isoxya Pro adds high availability, error recovery, and horizontal scaling. It is available on-premises for installation on Kubernetes, or in the cloud via SaaS subscription. Isoxya Pro is available via commercial licence. The API is able to scale crawlers, processors, and streamers. Support or custom development is available from Isoxya's creator.
Isoxya | Isoxya Pro | |
---|---|---|
Intended audience | Home · Hobbyist · R&D | Business · Agency |
API | ✅ | ✅ |
CLI scripts | ✅ | ✅ |
Extensible via plugins | ✅ | ✅ |
Web crawls | ✅ | ✅ |
External database | ❌ SQLite | ✅ PostgreSQL |
Cache | ❌ | ✅ Redis |
Message broker | ❌ | ✅ RabbitMQ |
Orchestrator | ❌ | ✅ Kubernetes |
High availability | ❌ | ✅ |
Error recovery | ❌ | ✅ |
Horizontal scaling | ❌ | ✅ |
Concurrent crawls | ❌ | ✅ |
Multi-process | ❌ | ✅ |
Custom user agents | ❌ | ✅ |
Custom rate limits | ❌ | ✅ |
Robots.txt compliance | ❌ | ✅ |
Cancellable crawls | ❌ | ✅ |
Limitable max pages | ❌ | ✅ |
Limitable max depth | ❌ | ✅ |
External links validation | ❌ | ✅ |
List crawls | ❌ | ✅ |
Resource management | ❌ | ✅ |
Scalable crawlers | ❌ | ✅ |
Scalable processors | ❌ | ✅ |
Scalable streamers | ❌ | ✅ |
Support | Community | |
Licence | BSD-3-Clause | Commercial |
Price | Free | Custom |
Download Free | Request Quote |