I was looking for downloading all the scan performed on all my web application in order to generate stats of vulnerabilities and check automatically if a scan generated an error or not.
To do so, I first use https://qualysapi.qualys.com/qps/rest/3.0/search/was/wasscan to get the full list of all the performed scans and their id
Then with a loop on the previous result, I use https://qualysapi.qualys.com/qps/rest/3.0/download/was/wasscan/<id> where id is the scan reference.
I'm doing this in python and after some results (between 2 and 20), I've the following error which happen:
URLError: <urlopen error [Errno 10054] An existing connection was forcibly closed by the remote host>
Between dowloading and parsing information, it takes approximatively 10minutes per scan to get the result and the script fails after 2 or more request with the above error. There is around 1900 scans on my account so if the script could do everything without an error it will be perfect
My internet connection is quite stable and I can't explain why this error happen.
Do you have any idea on this error?
Do I use the good API functions?
Is it normal that it takes that much time for every scan?
Is it possible to disable the vulnerability details in this kind of scan report?
Thanks in advance
The urls you are using are correct. The first returns the list of Web App Scans based on your the filters you provided in your API request. The second will return the XML content of a scan.
Could you please try downloading all the XML scan results and, once all the scans are downloaded, launch the parsing process on each of them. That way you could separate fetching the files from parsing them.
Thanks for your fast answer. I just implemented it, so to download all the XML report and then parse them. But the same error occur with the fourth report.
Also, the files created are around 875000 lines and about 130MB... Is it possible to ask for less details in these reports? As 1900 reports of about 100MB will be a bit too much...
Thanks in advance
In fact it will be too much to download.
Unfortunately, todayn the Scan download API doesn't provide options for now. We may consider this case as a feature to implement for the future releases.
Quick question: what's the size of one of those xml reports when you zip it?
Zipped, it size is around 3-4MB. But I haven't found an option where you can specify the output of the format in the API....
Should I open a feature request for requesting less detailled reports?
130 Mb for an XML scan results looks definitely too big, even for an extremely vulnerable web application.
Brice, I'll contact you to investigate deeper the issue.
Otherwise, to answer your question "Is it possible to disable the vulnerability details in this kind of scan report?", that's an option we can indeed consider if you're just interested by the QIDs and main information like vulnerable URL and param.
Let me add this to our roadmap for the scan API.