Palo Alto Networks PAN-OS v8.1-10.0 Elastic Stack v7.x Configuration

There are several Palo Alto projects for ELK but most seem to be vacated with no updates in the past year. Also could not find any with PAN-OS 9 or 10+ expanded logs (SD-Wan).

Background

Update existing projects to CIM and PAN 10.0 (Will work with PAN-OS 8+) Initial Updates from other projects:

• Support of ELK v7.8
• Added new fields for traffic logs that started with PAN-OS 9.1 and 10.0
• Changed attribute names from the default PA field names to Common Information Model (CIM) where applicable.
  Allow you to import CIM Traffic and Threat visualizations
• Added DNS filter to provide hostnames not just the IP including DNSMASQ install reference
• Converted all import objects to ndjson (ELK is migrating away from json)
• Import of indexes so visualizations match index UUID
• Added prune to pipeline to get ride of normaly null and duplicated fields.
  Also helps when PA adds a new log field, no more COL# fields
• Created panos-undefined index to capture logging of other types
• Added destination map to traffic dashboard

If you like the visualizations and dasboards, please buy me a coffee so I can keep going https://www.patreon.com/gauthig

Tutorial

This project was built on Ubuntu 20.04, and adding the ELK repositories so that the ELK stack stays current. Instructions are provided for this OS base, ELK setup.

1 - ELK install using repositories

2 - Install dnsmasq

3 - After ELK Install (or if ELK already exists)

Once you have ELK up and running start here

• Download these folders as follows
  • elk-pipeline - files that need to be on the elk server
  • gui-import - files that will be imported via the kibana web gui

3.1 Edit 'pan-os.conf'

• Set your timezone correctly (Very important), also set you local server timezone so it is not UTC
• RAW Log The RAW output from the Palo Alto is saved in each document in the message field. This is required if you are on a PCI or other regulated firewall. This field is not parsed or indexed. If you want to save space and don't need raw message uncomment this section to not store the non-parsed raw syslog (Optional):
  # mutate {
  # # Original message has been fully parsed, so remove it.
  # remove_field => [ "message" ]
  # }
  

3.2 Copy files to your server

Copy pan-os.conf to your conf directory. For Ubuntu/Debian this is "/etc/logstash/conf.d/

3.3 Install the index template (adds GeoIP for maps and optimizes other fields)

• Run this command from the same directory where you put panos-template.json
• If running curl from another node please put the correct server IP and ensure port 9200 is open on the network (not a secure practice)

curl -XPUT http://127.0.0.1:9200/_template/panos-template?pretty -H 'Content-Type: application/json' -d @panos-template.json

3.4 Import the saved object files (in this order)

log into your kibana interface and go to the saved objects page

http://<yourkibana DNS or IP>:5601/app/kibana#/management/kibana/objects

Click on import and select each import file in this order 1-index.ndjson 2-visualizations.ndjson 3-dashboard.ndjson 4-maps.ndjson

3.5 Restart Elastic Search & LogStash

sudo systemctl restart elasticsearch.service sudo systemctl restart logstash.service

4 - PaloAlto Setup

• Configure your PANW Firewall(s) or Panorama to send syslog messages to your Elastic Stack server
• Use port 5514
• Ensure that your firewall generates at least one traffic, threat, system & config syslog entry each
• Traffic will be generated by just going to a web site (make sure you setup logging for your policies).
• You may have to trigger a threat log entry. Follow this guide from Palo Alto for instructions
• After committing to set your syslog server, you will need to do another commit (any change) to actually send a config log message. Try changing the order of a rule and committing it.

References

---

Credit and Contributions I have found several older OpenSource GitHub projects on Palo Alto to Elk setups and whish to thank the following early developers.
shadow-box - (https://github.com/shadow-box/Palo-Alto-Networks-ELK-Stack) sm-biz - (https://github.com/sm-biz/paloalto-elasticstack-viz)

---