Job description
1.
The problem involves only the Chrome browser and affects the browser itself, and appeared long after the development of a fully functional scraper. The web portal uses CloudFlare, which, if the user agent is swapped, returns a 403 error after interacting with the login module.
If the ua is changed only after logging into a regular Chrome instance, you can navigate the entire portal as a logged-in user. When you log out (with the changed ua) it triggers a 403.
The 403 is triggered when the ua is swapped:
- Selenium Chromedriver
- Chrome UA Spoofer plugin
- Modheader plugin
403 is not triggered, and portal login runs smoothly when the ua is swapped within Developer Tools in a regular browser.
The 403 is triggered in Selenium Chromedriver if the scraper is started with the default ua (obsolete for the portal), swapped later by the hitherto successful Developer Tools.