Basic Parsing with TextFSM

I have used parsing in other projects. I used Genie and TextFSM to parse Cisco output in order to get specific details.
TextFSM in Project
Genie in Project
Here I want to take a closer look at creating a TextFSM template. As I could not easily find a TextFSM template for the Nexus command “show ip mroute” I have created one.
As a base I used the IOS template that can be found on the Network to Code Git.

TextFSM Templates

Below I have a basic script that will use a cisco IOS multicast routing table template to parse the results of “show ip mroute”.
GitHub link

The template I used for this was cisco_ios_show_ip_mroute.textfsm. I saved this as a file in the same location as the script.


import textfsm
import os

basedir = os.path.abspath(os.path.dirname(__file__)) 

mrouteOutputIOS = '''
IP Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,
       L - Local, P - Pruned, R - RP-bit set, F - Register flag,
       T - SPT-bit set, J - Join SPT, M - MSDP created entry, E - Extranet,
       X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,
       U - URD, I - Received Source Specific Host Report,
       Z - Multicast Tunnel, z - MDT-data group sender,
       Y - Joined MDT-data group, y - Sending to MDT-data group,
       G - Received BGP C-Mroute, g - Sent BGP C-Mroute,
       N - Received BGP Shared-Tree Prune, n - BGP C-Mroute suppressed,
       Q - Received BGP S-A Route, q - Sent BGP S-A Route,
       V - RD & Vector, v - Vector, p - PIM Joins on route,
       x - VxLAN group
Outgoing interface flags: H - Hardware switched, A - Assert winner, p - PIM Join
 Timers: Uptime/Expires
 Interface state: Interface, Next-Hop or VCD, State/Mode

(*, 224.0.1.40), 1d07h/00:02:49, RP 10.1.1.1, flags: SL
  Incoming interface: Null, RPF nbr 0.0.0.0
  Outgoing interface list:
    GigabitEthernet0/1, Forward/Sparse, 1d07h/00:02:33

'''

with open(basedir +'/ios_mroute.textfsm') as template:
    fsm = textfsm.TextFSM(template)
    result = fsm.ParseText(mrouteOutputIOS)

print(fsm.header)
print(result)

import textfsm

import os

basedir = os.path.abspath(os.path.dirname(__file__))

mrouteOutputIOS = '''

IP Multicast Routing Table

Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,

L - Local, P - Pruned, R - RP-bit set, F - Register flag,

T - SPT-bit set, J - Join SPT, M - MSDP created entry, E - Extranet,

X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,

U - URD, I - Received Source Specific Host Report,

Z - Multicast Tunnel, z - MDT-data group sender,

Y - Joined MDT-data group, y - Sending to MDT-data group,

G - Received BGP C-Mroute, g - Sent BGP C-Mroute,

N - Received BGP Shared-Tree Prune, n - BGP C-Mroute suppressed,

Q - Received BGP S-A Route, q - Sent BGP S-A Route,

V - RD & Vector, v - Vector, p - PIM Joins on route,

x - VxLAN group

Outgoing interface flags: H - Hardware switched, A - Assert winner, p - PIM Join

Timers: Uptime/Expires

Interface state: Interface, Next-Hop or VCD, State/Mode

(*, 224.0.1.40), 1d07h/00:02:49, RP 10.1.1.1, flags: SL

Incoming interface: Null, RPF nbr 0.0.0.0

Outgoing interface list:

GigabitEthernet0/1, Forward/Sparse, 1d07h/00:02:33

'''

with open(basedir +'/ios_mroute.textfsm') as template:

fsm = textfsm.TextFSM(template)

result = fsm.ParseText(mrouteOutputIOS)

print(fsm.header)

print(result)

The output of this script is below. We can see it has picked out the key parts of the command output.


['MULTICAST_SOURCE_IP', 'MULTICAST_GROUP_IP', 'UP_TIME', 'EXPIRATION_TIME', 'RENDEZVOUS_POINT', 'FLAGS', 'INCOMING_INTERFACE', 'REVERSE_PATH_FORWARDING_NEIGHBOUR_IP', 'REGISTERING', 'OUTGOING_INTERFACE', 'FORWARD_MODE', 'OUTGOING_MULTICAST_UP_TIME', 'OUTGOING_MULTICAST_EXPIRATION_TIME']
[['*', '224.0.1.40', '1d07h', '00:02:49', '10.1.1.1', 'SL', 'Null', '0.0.0.0', '', ['GigabitEthernet0/1'], ['Forward/Sparse'], ['1d07h'], ['00:02:33']]]

['MULTICAST_SOURCE_IP', 'MULTICAST_GROUP_IP', 'UP_TIME', 'EXPIRATION_TIME', 'RENDEZVOUS_POINT', 'FLAGS', 'INCOMING_INTERFACE', 'REVERSE_PATH_FORWARDING_NEIGHBOUR_IP', 'REGISTERING', 'OUTGOING_INTERFACE', 'FORWARD_MODE', 'OUTGOING_MULTICAST_UP_TIME', 'OUTGOING_MULTICAST_EXPIRATION_TIME']

[['*', '224.0.1.40', '1d07h', '00:02:49', '10.1.1.1', 'SL', 'Null', '0.0.0.0', '', ['GigabitEthernet0/1'], ['Forward/Sparse'], ['1d07h'], ['00:02:33']]]

Creating a TextFSM Template

As I said above I was not able to locate a Nexus template and so have created my own. Creating this is a lot of Regex. I used a number of resources to help me create this template. The main resource was the IOS template as a guide.
This is a lot of regex, it is important that you understand regex. Use the links below as a reference point as I’m not going to explain the difference between \s and \S etc.

Here is my TextFSM template. I kept several things from the IOS template and deleted what I did not know. As there are only a few lines in the Nexus output I’ll go into some detail.

Nexus “show ip mroute” output


IP Multicast Routing Table for VRF "default"

(*, 232.0.0.0/8), uptime: 3d00h, pim ip
  Incoming interface: Null, RPF nbr: 0.0.0.0
  Outgoing interface list: (count: 0)


(*, 239.0.1.2/32), uptime: 00:00:26, pim ip
  Incoming interface: loopback0, RPF nbr: 10.1.10.1
  Outgoing interface list: (count: 1)
    Ethernet1/2, uptime: 00:00:26, pim


(172.16.2.200/32, 239.0.1.2/32), uptime: 00:00:29, pim ip
  Incoming interface: Ethernet1/1, RPF nbr: 10.1.1.1, internal
  Outgoing interface list: (count: 0)


(*, 239.255.255.250/32), uptime: 1d08h, pim ip
  Incoming interface: loopback0, RPF nbr: 10.1.10.1
  Outgoing interface list: (count: 2)
    Ethernet1/2, uptime: 1d08h, pim
    Ethernet1/1, uptime: 1d08h, pim

IP Multicast Routing Table for VRF "default"

(*, 232.0.0.0/8), uptime: 3d00h, pim ip

Incoming interface: Null, RPF nbr: 0.0.0.0

Outgoing interface list: (count: 0)

(*, 239.0.1.2/32), uptime: 00:00:26, pim ip

Incoming interface: loopback0, RPF nbr: 10.1.10.1

Outgoing interface list: (count: 1)

Ethernet1/2, uptime: 00:00:26, pim

(172.16.2.200/32, 239.0.1.2/32), uptime: 00:00:29, pim ip

Incoming interface: Ethernet1/1, RPF nbr: 10.1.1.1, internal

Outgoing interface list: (count: 0)

(*, 239.255.255.250/32), uptime: 1d08h, pim ip

Incoming interface: loopback0, RPF nbr: 10.1.10.1

Outgoing interface list: (count: 2)

Ethernet1/2, uptime: 1d08h, pim

Ethernet1/1, uptime: 1d08h, pim

TextFSM Template for Nexus


Value Filldown VRF_NAME (\S+)
Value Required MULTICAST_SOURCE_IP (\*|(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\S\d+))
Value Required MULTICAST_GROUP_IP (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\S\d+)
Value Required UP_TIME (\S+?)
Value Required INCOMING_INTERFACE (\S+)
Value Required REVERSE_PATH_FORWARDING_NEIGHBOUR_IP (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})
Value List OUTGOING_INTERFACE_COUNT (\S+)
Value List OUTGOING_INTERFACE (\S+)
Value List OUTGOING_MULTICAST_UP_TIME (\S+)


Start
  # First line
  ^IP\s+Multicast\s+(?:Forwarding|Routing)\sTable\sfor\sVRF\s\"${VRF_NAME}\"
  ^\((\*|\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\S\d+),\s(\*|\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\S\d+)\) -> Continue.Record
  ^\(${MULTICAST_SOURCE_IP},\s${MULTICAST_GROUP_IP}\),\suptime:\s${UP_TIME}\,
  ^\s+Incoming\sinterface:\s${INCOMING_INTERFACE},\sRPF\snbr:\s${REVERSE_PATH_FORWARDING_NEIGHBOUR_IP}
  ^\s+Outgoing\s+interface\s+list:\s\(count:\s${OUTGOING_INTERFACE_COUNT}\)
  ^\s+${OUTGOING_INTERFACE},\suptime:\s${OUTGOING_MULTICAST_UP_TIME}
  # Capture time-stamp if vty line has command time-stamping turned on
  ^Load\s+for\s+
  ^Time\s+source\s+is
  ^. -> Error

Value Filldown VRF_NAME (\S+)

Value Required MULTICAST_SOURCE_IP (\*|(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\S\d+))

Value Required MULTICAST_GROUP_IP (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\S\d+)

Value Required UP_TIME (\S+?)

Value Required INCOMING_INTERFACE (\S+)

Value Required REVERSE_PATH_FORWARDING_NEIGHBOUR_IP (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})

Value List OUTGOING_INTERFACE_COUNT (\S+)

Value List OUTGOING_INTERFACE (\S+)

Value List OUTGOING_MULTICAST_UP_TIME (\S+)

Start

# First line

^IP\s+Multicast\s+(?:Forwarding|Routing)\sTable\sfor\sVRF\s\"${VRF_NAME}\"

^$(\*|\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\S\d+),\s(\*|\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\S\d+)$ -> Continue.Record

^$${MULTICAST_SOURCE_IP},\s${MULTICAST_GROUP_IP}$,\suptime:\s${UP_TIME}\,

^\s+Incoming\sinterface:\s${INCOMING_INTERFACE},\sRPF\snbr:\s${REVERSE_PATH_FORWARDING_NEIGHBOUR_IP}

^\s+Outgoing\s+interface\s+list:\s$count:\s${OUTGOING_INTERFACE_COUNT}$

^\s+${OUTGOING_INTERFACE},\suptime:\s${OUTGOING_MULTICAST_UP_TIME}

# Capture time-stamp if vty line has command time-stamping turned on

^Load\s+for\s+

^Time\s+source\s+is

^. -> Error

The TextFSM template is split into two parts. The top being all the values, these are the headings or what will be the dictionary keys. The values are variables so we know what value is each part of the plain text.
The second part is for the definitions. This is what is looking line by line in the text to match what we have. This references the first parts values to identify what is what.

For example there are two “uptime” references in the ouput. How does it know which is the uptime for the multicast source and which is the uptime for the outgoing interface?
Line 4 is the value for multicast group uptime and is referenced on line 13
Line 8 is the value for the outgoing multicast interface uptime and is referenced on line 16.

I’ll go in depth here for how it works by picking out some lines.

Example 1


^(${MULTICAST_SOURCE_IP},\s${MULTICAST_GROUP_IP}),\suptime:\s${UP_TIME}\,

^(${MULTICAST_SOURCE_IP},\s${MULTICAST_GROUP_IP}),\suptime:\s${UP_TIME}\,

The above line references three variables; MULTICAST_SOURCE_IP, MULTICAST_GROUP_IP and UP_TIME.


Value MULTICAST_SOURCE_IP (*|(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}\S\d+))
Value MULTICAST_GROUP_IP (\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}\S\d+)
Value UP_TIME (\S+?)

Value MULTICAST_SOURCE_IP (*|(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}\S\d+))

Value MULTICAST_GROUP_IP (\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}\S\d+)

Value UP_TIME (\S+?)

This is the first line of the output that gives multicast routing details. Examples are below


(*, 232.0.0.0/8), uptime: 3d00h, pim ip
Or
(172.16.2.200/32, 239.0.1.2/32), uptime: 00:00:29, pim ip

(*, 232.0.0.0/8), uptime: 3d00h, pim ip

(172.16.2.200/32, 239.0.1.2/32), uptime: 00:00:29, pim ip

Example 2


  ^\s+Incoming\sinterface:\s${INCOMING_INTERFACE},\sRPF\snbr:\s${REVERSE_PATH_FORWARDING_NEIGHBOUR_IP}

^\s+Incoming\sinterface:\s${INCOMING_INTERFACE},\sRPF\snbr:\s${REVERSE_PATH_FORWARDING_NEIGHBOUR_IP}


Value INCOMING_INTERFACE (\S+)
Value REVERSE_PATH_FORWARDING_NEIGHBOUR_IP (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})

Value INCOMING_INTERFACE (\S+)

Value REVERSE_PATH_FORWARDING_NEIGHBOUR_IP (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})

This line is for the incoming interface, where the multicast traffic is being received. It references two variables in the same way that the example does. I have used regexr.com to demonstrate how to grab the vales.

On the line we are first looking for “Incoming Interface:” and picking out the value after this. In this example it will be Null.

The next part is to view the incoming interface variable. The interface I am looking for will be characters that go upto a comma after the word.
“Null,” or “Ethernet1/2,”.


Value INCOMING_INTERFACE (\S+)
${INCOMING_INTERFACE},

Value INCOMING_INTERFACE (\S+)

${INCOMING_INTERFACE},

The last part to this line is to find the RPF neighbour IP. I have a variable for that. It is looking for an IP address.


Value REVERSE_PATH_FORWARDING_NEIGHBOUR_IP (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})
\sRPF\snbr:\s${REVERSE_PATH_FORWARDING_NEIGHBOUR_IP}

Value REVERSE_PATH_FORWARDING_NEIGHBOUR_IP (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})

\sRPF\snbr:\s${REVERSE_PATH_FORWARDING_NEIGHBOUR_IP}

Now to get the IP address. If there were multiple IP addresses in the text file. Regex would locate all the IP addresses. This is why it is important to use the variable values to dictate which IP or other multiple mnatching element is in a particular location.

Example 3

This part explains the lists. For the outgoing interfaces there may be multiple interfaces. So there needs to be a way to gather them all up along with other details.
In the TextFSM template the value can have the “list” keyword before the variable value name. This makes the output store the matching items as a list.

Notice below the interfaces and uptime values are in their own lists.


'VRF_NAME', 'MULTICAST_SOURCE_IP', 'MULTICAST_GROUP_IP', 'UP_TIME', 'INCOMING_INTERFACE', 'REVERSE_PATH_FORWARDING_NEIGHBOUR_IP', 'OUTGOING_INTERFACE_COUNT', 'OUTGOING_INTERFACE', 'OUTGOING_MULTICAST_UP_TIME']
['default', '172.16.2.200/32', '239.0.1.2/32', '15:47:17', 'Vlan20', '172.16.2.200', ['1'], ['Vlan16'], ['15:47:13,']]

'VRF_NAME', 'MULTICAST_SOURCE_IP', 'MULTICAST_GROUP_IP', 'UP_TIME', 'INCOMING_INTERFACE', 'REVERSE_PATH_FORWARDING_NEIGHBOUR_IP', 'OUTGOING_INTERFACE_COUNT', 'OUTGOING_INTERFACE', 'OUTGOING_MULTICAST_UP_TIME']

['default', '172.16.2.200/32', '239.0.1.2/32', '15:47:17', 'Vlan20', '172.16.2.200', ['1'], ['Vlan16'], ['15:47:13,']]

The rest of the regex is the similar to how it works in the other examples.

Textfsm with Scrapli

For this I will install Scrapli in full, this will install Scrapli along with other components such as TextFSM.


python -m pip install 'scrapli[full]'

python -m pip install 'scrapli[full]'

TextFSM Template in Scrapli

I have created a cut down version of the script I will use in the Scrapli and Flask project for testing purposes. This script will login to a Nexus switch and run the command “show ip mroute”. The output will be parsed by my own NXOS template.
Full script can be found in GitHub.

The output of the “show ip mroute” command and script output is below. My original template was overwriting values.
So if there were multiple multicast groups, it would be the last group that was parsed and output. This was resolved by using the options and specifically telling TextFSM where to start recording values from.
In respect to the mroute table this is from the multicast group IP line.


mcast_nexus1# sh ip mroute
IP Multicast Routing Table for VRF "default"

(*, 232.0.0.0/8), uptime: 3d13h, pim ip
  Incoming interface: Null, RPF nbr: 0.0.0.0
  Outgoing interface list: (count: 0)


(172.16.2.200/32, 239.0.1.2/32), uptime: 13:18:18, ip pim
  Incoming interface: Vlan20, RPF nbr: 172.16.2.200
  Outgoing interface list: (count: 1)
    Vlan16, uptime: 13:18:15, pim


(*, 239.255.255.250/32), uptime: 1d21h, igmp ip pim
  Incoming interface: Ethernet1/1, RPF nbr: 10.1.1.2
  Outgoing interface list: (count: 1)
    Vlan20, uptime: 1d21h, igmp

mcast_nexus1# sh ip mroute

IP Multicast Routing Table for VRF "default"

(*, 232.0.0.0/8), uptime: 3d13h, pim ip

Incoming interface: Null, RPF nbr: 0.0.0.0

Outgoing interface list: (count: 0)

(172.16.2.200/32, 239.0.1.2/32), uptime: 13:18:18, ip pim

Incoming interface: Vlan20, RPF nbr: 172.16.2.200

Outgoing interface list: (count: 1)

Vlan16, uptime: 13:18:15, pim

(*, 239.255.255.250/32), uptime: 1d21h, igmp ip pim

Incoming interface: Ethernet1/1, RPF nbr: 10.1.1.2

Outgoing interface list: (count: 1)

Vlan20, uptime: 1d21h, igmp


host: Scrapli Driver 172.16.1.115:22, OS: nxos
I am Nexus and running Nexus command
{'vrf_name': 'default', 'multicast_source_ip': '*', 'multicast_group_ip': '232.0.0.0/8', 'up_time': '3d16h', 'incoming_interface': 'Null', 'reverse_path_forwarding_neighbour_ip': '0.0.0.0', 'outgoing_interface_count': ['0'], 'outgoing_interface': [], 'outgoing_multicast_up_time': []}
{'vrf_name': 'default', 'multicast_source_ip': '172.16.2.200/32', 'multicast_group_ip': '239.0.1.2/32', 'up_time': '15:55:47', 'incoming_interface': 'Vlan20', 'reverse_path_forwarding_neighbour_ip': '172.16.2.200', 'outgoing_interface_count': ['1'], 'outgoing_interface': ['Vlan16'], 'outgoing_multicast_up_time': ['15:55:44,']}
{'vrf_name': 'default', 'multicast_source_ip': '*', 'multicast_group_ip': '239.255.255.250/32', 'up_time': '2d00h', 'incoming_interface': 'Ethernet1/1', 'reverse_path_forwarding_neighbour_ip': '10.1.1.2', 'outgoing_interface_count': ['1'], 'outgoing_interface': ['Vlan20'], 'outgoing_multicast_up_time': ['2d00h,']}

host: Scrapli Driver 172.16.1.115:22, OS: nxos

I am Nexus and running Nexus command

{'vrf_name': 'default', 'multicast_source_ip': '*', 'multicast_group_ip': '232.0.0.0/8', 'up_time': '3d16h', 'incoming_interface': 'Null', 'reverse_path_forwarding_neighbour_ip': '0.0.0.0', 'outgoing_interface_count': ['0'], 'outgoing_interface': [], 'outgoing_multicast_up_time': []}

{'vrf_name': 'default', 'multicast_source_ip': '172.16.2.200/32', 'multicast_group_ip': '239.0.1.2/32', 'up_time': '15:55:47', 'incoming_interface': 'Vlan20', 'reverse_path_forwarding_neighbour_ip': '172.16.2.200', 'outgoing_interface_count': ['1'], 'outgoing_interface': ['Vlan16'], 'outgoing_multicast_up_time': ['15:55:44,']}

{'vrf_name': 'default', 'multicast_source_ip': '*', 'multicast_group_ip': '239.255.255.250/32', 'up_time': '2d00h', 'incoming_interface': 'Ethernet1/1', 'reverse_path_forwarding_neighbour_ip': '10.1.1.2', 'outgoing_interface_count': ['1'], 'outgoing_interface': ['Vlan20'], 'outgoing_multicast_up_time': ['2d00h,']}

Textfsm Options

TextFSM options documentation can be found in this link.
I have used; Filldown, Required and List for the values. I have also used Continue and Record for the definitions.

To get a full documentation explaination of each option use this link. I will discuss how they helped me.

Filldown: This allows the VRF name to be added to each multicast list. Without this option the “vrf_name” value would be blank, and in a list by itself.

Required: This states which values are required to have data in them. Without this my first list would contain the VRF name and everything else would be empty.

List: This permits a list of items such as interfaces or uptimes. I have used this for the outgoing multicast interfaces.

Continue: This was in the IOS template. It is used after the definition that identifies the multicast source and group. Without this I don’t get any parsed output.

Record: This is on the same line as “Continue”. It was also in the IOS template I have used from NTC. Without this option, only the last multicast group is recorded. All of the other values are overwritten.

TextFSM Templates

Creating a TextFSM Template

TextFSM Template for Nexus

Example 1

Example 2

Example 3

Textfsm with Scrapli

TextFSM Template in Scrapli

Textfsm Options

Leave a Comment Cancel Reply