Monitoring Appliance Capacity & Utilization

Document created by Busby on Jan 24, 2017Last modified by Busby on Feb 8, 2017
Version 4Show Document
  • View in full screen mode

Ever wish you could monitor the performance of all your scans across all of your appliances in Qualys; well you can with a little scripting of the Qualys API, and creativity.  I am going to give you hopefully a few ideas of what you can do with some code.  If you have questions at all please let me know and I will try to answer to the best of my ability.

 

First; the problem.  I wanted to get a rough idea of how all my appliances are being utilized.  This was important for when I want more money for scanners I need to answer why I need more.  Are they being overloaded? things of this nature would be needed by management and I might use the information to see if my scans are balanced or are they favoring scanners.  I also have multiple networks and some times outages that affects scans so I wanted all of this information.  Maybe even send the data to splunk for further correlation.

 

So step one was to get a look at the appliance API and see what I get for data on the appliances.  To get the appliance information an example API call would be:

curl.exe --silent --tlsv1 --insecure --compressed --header "X-Requested-With: appliancestatus" --cookie ".\appliancestatus.cookies" --data "action=list&output_mode=full&scan_detail=1" "https://qualysapi.qualys.com:443/api/2.0/fo/appliance/" > "appliancestatus.xml"

 

Now aside from the standard API I call I would like to call out some of the parameters to the appliance API, only a couple.

Action = List   - Just giving me the information for Listing; no editing of the appliance going on.

output_mode = full - there is also brief but this gave me a LOT more data (more on that in a minute)

scan_detail=1 - do you want the detail of the scans (Yes=1, no=0)

 

Now here is a sample of XML that you get back (yes data is fake)

BRIEF Mode would give you this:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE APPLIANCE_LIST_OUTPUT SYSTEM "https://qualysapi.qualys.com/api/2.0/fo/appliance/appliance_list_output.dtd">
<APPLIANCE_LIST_OUTPUT>
  <RESPONSE>
    <DATETIME>2016-12-16T13:47:50Z</DATETIME>
    <APPLIANCE_LIST>
      <APPLIANCE>
        <ID>999</ID>
        <UUID>db6de37b-0cf5-e5bf-80dc-be96b2e0e566</UUID>
        <NAME>scanner_01</NAME>
        <NETWORK_ID>0</NETWORK_ID>
        <SOFTWARE_VERSION>2.6</SOFTWARE_VERSION>
        <RUNNING_SLICES_COUNT>5</RUNNING_SLICES_COUNT>
        <RUNNING_SCAN_COUNT>0</RUNNING_SCAN_COUNT>
        <STATUS>Online</STATUS>
      </APPLIANCE>
      <APPLIANCE>
        <ID>109891</ID>
        <UUID>b9faecc1-58b2-d7a0-8284-d95deeabefc7</UUID>
        <NAME>scanner_02</NAME>
        <NETWORK_ID>0</NETWORK_ID>
        <SOFTWARE_VERSION>2.6</SOFTWARE_VERSION>
        <RUNNING_SLICES_COUNT>5</RUNNING_SLICES_COUNT>
        <RUNNING_SCAN_COUNT>0</RUNNING_SCAN_COUNT>
        <STATUS>Online</STATUS>
      </APPLIANCE>

    </APPLIANCE_LIST>
  </RESPONSE>
</APPLIANCE_LIST_OUTPUT>

 

FULL MODE will give you this type of output

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE APPLIANCE_LIST_OUTPUT SYSTEM "https://qualysapi.qualys.com/api/2.0/fo/appliance/appliance_list_output.dtd">
<APPLIANCE_LIST_OUTPUT>
  <RESPONSE>
    <DATETIME>2016-12-16T13:47:21Z</DATETIME>
    <APPLIANCE_LIST>
      <APPLIANCE>
        <ID>51469</ID>
        <UUID>db6de37b-0cf5-e5bf-80dc-be96b2e0e566</UUID>
        <NAME>scanner_01</NAME>
        <NETWORK_ID>0</NETWORK_ID>
        <SOFTWARE_VERSION>2.6</SOFTWARE_VERSION>
        <RUNNING_SLICES_COUNT>5</RUNNING_SLICES_COUNT>
        <RUNNING_SCAN_COUNT>0</RUNNING_SCAN_COUNT>
        <STATUS>Online</STATUS>
        <MODEL_NUMBER>QGSA-3120-A1</MODEL_NUMBER>
        <SERIAL_NUMBER>23133</SERIAL_NUMBER>
        <ACTIVATION_CODE>20016248420429</ACTIVATION_CODE>
        <INTERFACE_SETTINGS>
          <INTERFACE>lan</INTERFACE>
          <IP_ADDRESS>136.250.128.38</IP_ADDRESS>
          <NETMASK>255.255.255.224</NETMASK>
          <GATEWAY>136.250.128.33</GATEWAY>
          <LEASE>Static</LEASE>
          <SPEED>1000</SPEED>
          <DUPLEX>Full</DUPLEX>
          <DNS>
            <DOMAIN>junk.COM</DOMAIN>
            <PRIMARY>10.1.1.23</PRIMARY>
            <SECONDARY>10.1.1.24</SECONDARY>
          </DNS>
        </INTERFACE_SETTINGS>
        <INTERFACE_SETTINGS>
          <SETTING>Disabled</SETTING>
          <INTERFACE>wan</INTERFACE>
          <IP_ADDRESS></IP_ADDRESS>
          <NETMASK>255.255.255.0</NETMASK>
          <GATEWAY>127.0.0.1</GATEWAY>
          <LEASE>Dynamic</LEASE>
          <SPEED>0</SPEED>
          <DUPLEX>Unknown</DUPLEX>
          <DNS>
            <PRIMARY>0.0.0.0</PRIMARY>
            <SECONDARY>0.0.0.0</SECONDARY>
          </DNS>
        </INTERFACE_SETTINGS>
        <PROXY_SETTINGS>
          <SETTING>Disabled</SETTING>
          <PROXY>
            <IP_ADDRESS>0.0.0.0</IP_ADDRESS>
            <PORT>0</PORT>
            <USER></USER>
          </PROXY>
        </PROXY_SETTINGS>
        <VLANS>
          <SETTING>Disabled</SETTING>
        </VLANS>
        <STATIC_ROUTES />
        <ML_LATEST>9.0.29-1</ML_LATEST>
        <ML_VERSION updated="yes">9.0.29-1</ML_VERSION>
        <VULNSIGS_LATEST>2.3.499-2</VULNSIGS_LATEST>
        <VULNSIGS_VERSION updated="yes">2.3.499-2</VULNSIGS_VERSION>
        <ASSET_GROUP_COUNT>207</ASSET_GROUP_COUNT>
        <ASSET_GROUP_LIST>
          <ASSET_GROUP>
            <ID>1713871</ID>
            <NAME><![CDATA[3rd Party Hosted IPs: Google]]></NAME>
          </ASSET_GROUP>
          <ASSET_GROUP>
            <ID>1546161</ID>
            <NAME><![CDATA[AaTC - DE0021/DE0027]]></NAME>
          </ASSET_GROUP>
          <ASSET_GROUP>
            <ID>1545740</ID>
            <NAME><![CDATA[AbTC - GB0045]]></NAME>
          </ASSET_GROUP>
          <ASSET_GROUP>
            <ID>1622607</ID>
            <NAME><![CDATA[AbTc License Server VMs]]></NAME>
          </ASSET_GROUP>

 

As you can see you get a LOT more data from the FULL mode. I do not use all of this today but my code is geared around this as the output to handle.  Next I did go through trying to determine what data points I wanted; after all I need to create a database table to store this; although if I wanted too a file would work too.

 

Here are the data points I collected.

    CREATE TABLE `qualysappliances` (
  `appliance_statusdatetime` datetime NOT NULL,
  `appliance_id` int(11) NOT NULL,
  `appliance_uuid` varchar(100) NOT NULL,
  `appliance_name` varchar(45) NOT NULL,
  `appliance_network_id` int(11) NOT NULL,
  `appliance_software` float NOT NULL,
  `appliance_slices` int(11) NOT NULL,
  `appliance_scans` int(11) NOT NULL,
  `appliance_status` varchar(45) NOT NULL,
  `appliance_capacityunits` int(11) NOT NULL,
  `appliance_heartbeatsmissed` int(11) NOT NULL,
  `appliance_ssconnection` varchar(15) NOT NULL,
  `appliance_last_connected` datetime NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='stores data about the status of the Qualys Appliances to be used for trending analysis.\nID = ""\n                UUID = ""\n                NAME = ""\n                NETWORK_ID = ""\n                SOFTWARE_VERSION = ""\n                RUNNING_SLICES_COUNT = ""\n                RUNNING_SCAN_COUNT = ""\n                STATUS = ""';

 

Now the statusdatetime above is basically just a time date stamp of when the data comes into the database; I am running this in the background about every 10 minutes or so.  Now you can extract and plot the information.

 

Some things to be aware of; to see if a scanner is "busy" you really need to define what busy means.  Especially if you have the slicing of scans enabled on your account.  You will see a scanner in the data with 0 slices but 1 scans or more.  Each appliance can handle more than 1 scan and more than 1 slice at a time; but only one appliance is in control of any one scan.

 

Think of this; Scanner A has 1 scan and 0 slices and Scanner B could have no scans and 5 slices.  So Scanner A is controlling the scan and has given 5 slices of the scan to scanner B.  Just so you know that is how it works.  Now the capacityunits will be the same number that is expressed as a percentage in Qualys of the number of unis left as I recall.  So what does this look like.

 

 

Now this is not all of it but now I can see all my scanners across all my networks and how busy they are, have they missed heartbeats, when did they last connect; etc...  This will allow me to build if I want real time charts of how busy my scanners are.  Once I correlate this with some of my big scans I should be able to give the information to management in a digestible form to make a decision.  Do we need more scanners? well if all the scanners have 0 capacity while doing our big scans then yes.  I also may be able to plot this over time and see that between the hours of 1:00am to 5:00am I am not really doing any scans.  Then I can adjust some of my schedules to go off during that time slice.

 

Thought a few of you might find this useful.  If you would like more information on how I am doing this in powershell please let me know.

 

jbleggett   suggested I graph this, here is what one graph may show you, the number of slices of scans over time by Appliance.

 

Graph of Slices of scans by Appliance over time.

David

3 people found this helpful

Attachments

    Outcomes