Job description
We need a crawler to collect external links from websites - this is urgent.
To be able to set: the maximum number of subpages scanned from a given domain,
what time does it scan the subpage in seconds, depth of download and simultaneous scanning of multiple domains. Gets the domain to scan with mysql and results to mysql.
After the successful first script, we will want to order more.
