When querying a firewall, what’s the best protocol to use? SNMP, SSH or API?

If you are looking to integrate Palo Alto firewalls as part of some automated system – scripts, central NOC, software-defined-whatever, etc. – you’d want to hear what we have to share. You should also read this post if you like learning about interesting technical aspects of the products you use.

As you may know, we have started supporting Palo Alto Networks (PANW) firewalls in our product late last year. We are currently developing new support and are working with large and small organizations throughout the globe. One interesting thing we’ve noticed that’s worth sharing is that PANW’s customers are very open to embracing new technologies. That is great for us 🙂

As we started building our support for PANW firewalls, we’ve looked at the different possible methods for retrieving data from the firewalls themselves. When reading this, keep in mind indeni is read-only and pulls data at fairly high intervals (to allow for real-time alerting) and so our requirements may be a bit different to yours. These are the three possible methods we looked at:

Quering a Firewall with SNMP:

The good old protocol from 1990. Still prevalent and supported by any network-attached device in the world. However, fairly limited in the amount and complexity of data that can be retrieved from the device.

Querying Firewall via SSH:

Our protocol of choice with many devices. It allows us to pull more data at the cost of needing to parse and understand that data, even if its structure changes from version to version. Usually we’ll go with this option if the API (see below) is not available or stable.

Querying a Firewall via API:

As things progress, more manufacturers provide APIs for querying their devices as well as making changes. Different manufacturers utilize different underlying communication protocols (HTTPS usually) and the API calls vary greatly. We also noticed that with some manufacturers, such as F5, the API is very different between major versions. In PANW’s case, it was amazingly easily for us to integrate with the API – kudos to the team for that!

Naturally – we started with the API. When a manufacturer provides an API, our immediate assumption is that it’s the recommended way of interacting with the device programmatically (after all, API stands for Application Program Interface). After some testing in our lab, and more testing with some of our PANW beta users, we’ve come to the conclusion we can’t rely solely on the API. The primary reason: CPU usage (on the management plane).

IMPORTANT NOTE: The graphs above show high CPU utilization due to the queries – that is because we were hammering the device with queries in order to generate significant difference. If you were to run API queries at far longer intervals the CPU utilization would be far less.

In the graphs above you can see a test we’ve run on one of our PANW devices in the lab (running PAN-OS 6.1.2). The “NO QUERIES” graph represents CPU usage when no queries are being done and serves as a baseline. Then, we ran the same queries through the three different methods. We tried to retrieve CPU information, network interface list and stats, etc. So the content that is being retrieved is the same, just the method varies.

It’s clear that the API is more CPU-intensive than the SNMP method. There may be a variety of reasons for this and it is not my place to speculate, but what it does show you is that when you build an automated system for doing something, you must be cognizant of its possible impact on the environment.

That’s why we run these tests.
That’s why we put safety mechanisms in place (indeni will disengage a device if its CPU utilization spikes, for example).
That’s why we do what we do.

As you may expect – we’ve decided to use a mix of SNMP, SSH and API in our support for PANW firewalls – to maximize the amount of information we can retrieve while protecting the firewall’s resources. If you haven’t given our Palo Alto Networks Firewall support a spin, now is a great time!

We have received requests to share the test script. So here it is:

• Parameters:
  • $device – the IP of the device.
  • $key – the API key (this is generated once at the beginning of the entire run to avoid wasting time / CPU on regenerating the key again and again)
• Important notes:
  • The script is run from a test server that interacts (makes SNMP GETs, HTTPS GETs, etc.) with the firewall. The script is NOT run on the firewall itself.
  • The test scripts (the ones that actually fetch the data) run endlessly without any “sleep” or wait. It means we’re hammering the firewall quite heavily, far more than anyone normally would.
• The script is actually two scripts running in parallel:
  • The first script captures the output of “snmpwalk -v2c -c public $device HOST-RESOURCES-MIB::hrProcessorLoad.1” every 5 seconds.
  • The second script (remember, runs in parallel to the first one) does one of the following:
    • For NO QUERIES – it does nothing.
    • For API it makes the following calls:
      • curl -s –insecure “https://$device/?type=op&cmd=<show><interface>all</interface></show>&key=$key
      • curl -s –insecure “https://$device/api/?type=op&cmd=<show><interface>ethernet1/1</interface></show>&key=$key”
      • curl -s –insecure “https://$device/api/?type=op&cmd=<show><interface>ethernet1/2</interface></show>&key=$key”
      • curl -s –insecure “https://$device/api/?type=op&cmd=<show><interface>ethernet1/3</interface></show>&key=$key”
      • curl -s –insecure “https://$device/api/?type=op&cmd=<show><interface>ethernet1/4</interface></show>&key=$key”
    • For SNMP it runs:
      • snmptable -v2c -c public $device IF-MIB::ifTable
    • For SSH it:
      • Logs in via SSH and then runs the following commands (on the firewall):
        • show interface all
        • show interface ethernet1/1
        • show interface ethernet1/2
        • show interface ethernet1/3
        • show interface ethernet1/4