Property:Has bot segment

Has bot segment is a Has type::Number and is assigned by Template:Website. The purpose of this property is to group Category:Websites into a small number of groups for bots to separately collect information for, parallelizing the statistics collection. As the number of websites increases above several hundred, User:Bumble Bee will operate better by doing multiple sites concurrently. Rather than introducing a lot of complexity with multithreaded programming in the bot, we instead just group the sites into multiple segments and the scripts can then deal with subsets of the sites concurrently, with multiple processes.

This property using a modulo of the  by the number of bots running. There are currently  bot segments, which is set by changing the value at Bot segments. The statistics bots then run in parallel via cron, with each run taking a different segment. These may also be offset in cron so they aren't running all at once. One approach using 6 segments would be to have two running in parallel every 5 minutes, repeating the loop every 15 minutes.

Currently the breakdown across  websites: