the journey of a million pages
… begins with a single request. Starting the crawl and discovering the site graph is typically the slowest part. The initial data can then be used to efficiently load crawling, processing, and streaming queues.
Isoxya is able to dynamically allocate resources and initiate crawls quickly, reaching high speeds even within the first 60 seconds. It also supports multi-concurrency channels for crawlers, processors, and streamers—for even the most demanding of workloads.
Here you can watch a small demo of Isoxya completing 401 pages end-to-end within 32 seconds. But it's possible to go far quicker.