Scrapli and Flask – Part 5: Additional Device Details

Automation, Cisco Networking, Lab /

In version2, I have made several changes to continue with this project

  • Added device details page for each device
  • Refresh database button for upto date device details
  • Refresh button for each device
  • Notification if the last refresh of device data is greater than 200 seconds
  • Also added is a device base template cofig generator. More details can be found in part 6

Code for the this current version2 can be found in GitHub

  • Add IP Addresses
  • Error Checking of IPs
  • New Inventory
  • No Device Details Yet
  • No Device Details Yet
  • No Device Details Yet
  • Port Details Gathered
  • Show Run Details Collected
  • Polling Time Added
  • Polling Time Notifies More than 200 Seconds

Making the Details Pages

This was quite a simple process to create the new individual pages for the network devices

I first needed to create a view that was expecting a variable of the hostname

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
 
@devices_blueprint.route('/<hostname>', methods = ['GET', 'POST'])
def device_info(hostname): # var hostname is required
    """
    Displays the details for the device
    Three tabs
    Tab1 basic info from devices_table
    Tab2 switchport info from device_details_table
    Tab3 show run output from device_details_table
    """
    
    # Get details to pass into template
    device_stats = DBcontroller()
    device_info = device_stats.get_device_basic_info(hostname)
    device_details = device_stats.get_device_details(hostname)
    print(device_info)
 
    if device_details != []:
        show_run = device_details[0][3].split('\n')
        lastupdate = device_details[0][8]
    else:
        show_run = "No Entry found, refresh data"
        lastupdate = "No Entry found, refresh data"
        device_details = "Empty"
 
    return render_template('detail.html', hostname=hostname, device_info=device_info, show_run=show_run, lastupdate=lastupdate, device_details=device_details)
 

The links on the page are created in the HTML template of the inventory page. I added them to the hostname of each device

0
1
2
 
                <td> <a href="{{ row.hostname }}">{{ row.hostname }}</a</td>
 

Lastly a detail.html page is created that is reused for each device

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
 
{% extends "base.html" %}
{% block content %}
<div class="jumbotron">
    <h1>{{ hostname }} Details</h1>
</div>
 
 
<div class="row justify-content-between align-items-right col px-md-3">
    <div class="col-lg-auto col px-md-5">
        <a href="details_{{hostname}}" id="info_button"><button class="btn btn-primary btn-lg">Refresh Single Device Details</button></a>
        <div class="col-lg-auto">
            <div id="f5_info" class="row col-lg-auto badge badge-info"></div>
        </div>
    </div>
    <div class="col-lg-auto">
       <div class="alert alert-primary">
          Last updated: <strong>{{ lastupdate }}</strong>
       </div>
    </div>
</div>
 
<br>
<br>
 
 
 
 
<div class="container">
    <div class="jumbotron">
       <ul class="nav nav-tabs">
          <li class="nav-item">
             <a class="nav-link active" data-toggle="tab" href="#quickstats">Basic Info</a>
          </li>
          <li class="nav-item">
             <a class="nav-link" data-toggle="tab" href="#portdetail">Port Info</a>
          </li>
          <li class="nav-item">
             <a class="nav-link" data-toggle="tab" href="#rawoutput">"Show Run" Raw Output</a>
          </li>
       </ul>
      
        <div id="myTabContent" class="tab-content">
            <div class="tab-pane active" id="quickstats">
                <div class="row">
                    <div class="col-lg-6">
                        <br>
                        <p><strong>Hostname</strong> {{ device_info[0][1] }}</p>
                        <p><strong>Model</strong> {{ device_info[0][4] }}</p>
                        <p><strong>Software Version:</strong> {{ device_info[0][3] }}</p>
                        <p><strong>OS Type:</strong> {{device_info[0][5] }}</p>
                    </div>
                    <div class="col-lg-6">
                        <br>
                        <p><strong>Management IP:</strong> {{device_info[0][2] }}</p>
                        <p><strong>Serial:</strong> {{device_info[0][6] }}</p>
                        <p><strong>Uptime:</strong> {{device_info[0][7] }}</p>
                    </div>
                </div>
            </div>
        
            <div class="tab-pane" id="portdetail">
                <div class="row">
                    <div class="col-lg-6">
                        <br>
                        {% if device_details == "Empty" %}
                            <p> <strong>Total Physical Ports: </strong>Refresh Data</p>
                            <p><strong>Total VLAN Interfaces: </strong>Refresh Data</p>
                            <p><strong>Ports Shutdown: </strong>Refresh Data</p>
                            <p><strong>Ports Up (Inc SVIs): </strong>Refresh Data</p>
                        {% else%}
                            <p> <strong>Total Physical Ports:</strong> {{device_details[0][4] }}</p>
                            <p><strong>Total VLAN Interfaces:</strong> {{ device_details[0][5] }}</p>
                            <p><strong>Ports Shutdown:</strong> {{ device_details[0][6] }}</p>
                            <p><strong>Ports Up (Inc SVIs):</strong> {{ device_details[0][7] }}</p>
                        {% endif %}      
                    </div>
                </div>
            </div>
 
      
            {% if show_run %}
 
                <div class="tab-pane" id="rawoutput">
                    <br>
                    <code>
                    <br>
                    {% if show_run == "No Entry found, refresh data" %}
                        <p>{{ show_run }}</p>
                    {% else %}
                        {% for line in show_run %}
                            <p>{{ line }}</p>
                        {% endfor %}
                    {% endif %}
                    </code>
                </div>
 
            {% else %}
            
                <div class="tab-pane" id="rawoutput">
                    <div class="row justify-content-between align-items-right col px-md-3">
 
                        <br>
                        <br>
 
                        <div class="col-lg-auto"></div>
                        <div class="col-lg-auto">
                           <div class="alert alert-primary">
                              Last updated: <strong>{{ lastupdate }}</strong>
                           </div>
                        </div>
                     </div>
 
 
                    <br>
                    <code>
                    <br>
                    <p>No show run yet. Please press refresh button</p>
                    </code>
                </div>
 
            {% endif %}
 
        </div>
    </div>
</div>
 
{% endblock %}
 

Gathering Show Commands

For the details page there are two additional commands that are run.
Show run
Show interface or show interfaces – depending on IOS or NXOS

The commands are all part of the same background process so we can technically add more. These commands are run and then parsed for specifics that I want. I have collected the show run, to output as raw output and also basic port details.
Multi threading is working on these commands also. I made some modifications so that from the views.py I am sending a string that is used to reference what I want to run from the threading function.
This allows all commands to be performed through the threading function in a repeatable and neat way.

Views.py

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
 
@devices_blueprint.route('/details_<hostname>', methods = ['GET', 'POST'])
def get_device_details(hostname): # var hostname is required
    """
    Get the specific device details
    Same as the view bg_get_info, but single device
    Get IP, convert so it is in a list and send to "get_detailed" function of background_processes.py
    Redirect back to device detail page
    """
 
    # Get details to pass into template
    device_stats = DBcontroller()
    device_info = device_stats.get_device_basic_info(hostname)
    ip = [device_info[0][2]]
 
    # Perform show run
    get_detailed = CiscoCommands()
    genie_list, error_ips = get_detailed.create_threads(ip, "get_detailed")
 
 
    # Add new details to DB
    device_details = DBcontroller()
 
    # Delete all data in table for hostname
    device_details.delete_device_details(hostname)
 
    # Add new details to database
    for device in genie_list:
        device_details.update_device_details( # Change this
                                            device['Hostname'],
                                            device['IP'],
                                            device['Show run'],
                                            device['Physical Ports'],
                                            device['Virtual Ports'],
                                            device['Disabled Ports'],
                                            device['Enabled Ports'],
                                            device['Updated']
                                            )
    device_details.close_db_conn()
 
    return redirect(url_for('devices.device_info',hostname=hostname))
 

background_processes.py

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
 
# Create Threads
    def create_threads(self, ips, user_command):
        """
        views.py sends the user_command
        """
        q = queue.Queue()
        genie_list = []
        error_ips = []
 
        # Take in the "command" from the view and convert to a functional call
        send_cmd = getattr(CiscoCommands(), user_command)
 
        
        # Create multithreading
        for thread_no in range(8):
            worker = Thread(target=send_cmd, args=(q, genie_list, error_ips, ), daemon=True)
            worker.start()
 
        # Start the threads
        for ip in ips:
            q.put(ip)
 
        q.join()
 
        return genie_list, error_ips
 
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
 
# Device Details function
    def get_detailed(self, q, genie_list, error_ips):
        while True:
            ip = q.get()
            
            try:
                host, os = CiscoCommands.conn(self, ip)
            except:
                error_ips.append(ip)
                q.task_done()
                break
 
            # perforn show run and get hostname from prompt
            output = host.send_command("show run")
            prompt = host.get_prompt()
 
            # Show interface details
            if os == "ios":
                shipintbr = host.send_command("show interfaces")
            elif os == "nxos":
                shipintbr = host.send_command("show interface")
            
            genie_parsed_response = shipintbr.genie_parse_output()
            # print(shipintbr.result)
            # print("\n")
            # print(genie_parsed_response)
            
 
            # Display number of port types
            physical_int = 0
            virtual_int = 0
 
            for interface in genie_parsed_response.keys():
                if "Ethernet" in interface:
                    physical_int +=1
                if "Vlan" in interface:
                    virtual_int +=1
 
            # Add to database
            print("\n")
            print(physical_int)
            print(virtual_int)
 
            
            
            
            # Display Port Status, finish this off
            shut_ports = 0
            noshut_ports = 0
            if os == "ios":
                for interface in genie_parsed_response.keys():
                    if genie_parsed_response[interface]['enabled'] == True:
                        noshut_ports +=1
                    elif genie_parsed_response[interface]['enabled'] == False:
                        shut_ports +=1
            elif os == "nxos":
                for interface in genie_parsed_response.keys():
                    if genie_parsed_response[interface]['admin_state'] == "up":
                        noshut_ports +=1
                    elif genie_parsed_response[interface]['admin_state'] == "down":
                        shut_ports +=1                
            
            # Add to database
            print("\n")
            print(f"Shutdown Ports: {shut_ports}")
            print(f"Enabled Ports: {noshut_ports}")
 
 
 
 
            # Get time
            now = datetime.now()
            timestamp = now.strftime("%B, %d, %Y %H:%M:%S")
 
            # Create list of dictionaries to be returned
            genie_dict = {"Hostname":prompt[:-1],
                        "IP":ip,
                        "Show run":output.result,
                        "Physical Ports": physical_int,
                        "Virtual Ports": virtual_int,
                        "Disabled Ports": shut_ports,
                        "Enabled Ports": noshut_ports,
                        "Updated":timestamp
                        }
            genie_list.append(genie_dict)
 
            q.task_done()
 

Refreshing All Devices or a Single Device

There are two options to pull all the device details into the database. Either refresh a single device, or refresh all. The single device may be refreshed from the device details page. If the device is new and there isn’t any details in the database the user is told. The time that the refresh has been performed is added to the database. This will be used later to determine if the past refresh of all devices was performed with 20 seconds of each other.
This is just a way to tell the user if what is in the database is recent.

Inventory HTML Page

0
1
2
3
4
5
6
7
 
    {% if lastupdate %}
        <div class="col-lg-auto">
            <div class="alert alert-primary">
            Last updated: <strong>{{ lastupdate }}</strong>
        </div>
    {% endif %}
 

Views.py – Calculates the time difference in seconds between the rows inthe database

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
 
@devices_blueprint.route('/inventory')
def inventory():
    """
    Basic page that pulls in the database table for devices_table
    Added more to allow to determine if the time between device updates in greater than 200 seconds
    """
 
    all_devices = Devices.query.all()
 
    # Get last updated for all devices, if they are greater than 200 seconds say so!
    all_devices_updated = DeviceInfo.query.all()
    lastupdate = "Never"
 
    if all_devices_updated:
        compare_times = []
        for times in all_devices_updated:
            compare_times.append(times.updated)
        
        # Sort times in order
        compare_times.sort(key=lambda date: datetime.strptime(date, "%B, %d, %Y %H:%M:%S"))
 
        # Calculate the difference in seconds between top and bottom number
        tfirst = datetime.strptime(compare_times[0], "%B, %d, %Y %H:%M:%S")
        tlast = datetime.strptime(compare_times[-1], "%B, %d, %Y %H:%M:%S")
        difference = tlast - tfirst
 
        # If difference is les than 200 seconds, display last update as last date/time
        if difference.seconds < 200:
            lastupdate = tlast
        else:
            lastupdate = "More than 200 seconds between polling"
            
    return render_template('inventory.html', all_devices=all_devices, lastupdate=lastupdate)
 

Leave a Comment

Your email address will not be published. Required fields are marked *