I've been occasionally experiencing issues with the v2 API when some of my automated scripts run overnight. They have been running overnight for over a year at this point. I'm using Python 3.6 with the Requests library. It uses session based login, but other scripts use basic authentication and still run into connection problems. I commonly see "Connection Aborted", "Connection Reset", and "Remote Host Closed Connection Without a Response". I've opened tickets a few times when it has occured, and the last time I was informed by Support it was due to an intermittent issue on Qualys' backend. Note that these connection issues occur regardless of which script, environment, or network is used.
So, since this is a recurring issue, I need to find a workaround. So far, I have adjusted my scripts to retry a number of times starting from the last successful page of data (id_min) before eventually closing out and sending an alert that it failed. This works, most of the time. I also wrote a script to verify the integrity of the stuff I download before I use it. Despite this, there has still been a few occasions where I was just unable to get a day's worth of data because there were just too many hiccups server side.
So, my question to y'all is has anyone else experienced these connection issues? If so, what have you done to mitigate them? Is there anything I can do client-side to avoid triggering these errors on the backend?