the journey of a million pages
… begins with a single request. Starting the crawl and discovering the site graph is typically the slowest part. The initial data can then be used to efficiently load crawling, processing, and streaming queues.
Isoxya is able to dynamically allocate resources and initiate crawls quickly, reaching high speeds even within the first 60 seconds. It also supports multi-concurrency channels for crawlers, processors, and streamers—for even the most demanding of workloads.
Here you can watch a small demo of Isoxya completing 410 pages end-to-end in 38 seconds. But it's possible to go far quicker.