Advanced topics¶
The following material is a deep-dive into Yangson, and is not necessarily representative of how one would perform manipulations in a production environment. Please refer to the other tutorials for a better picture of Rosetta’s intended use. Keep in mind that the key feature of Yangson is to be able to manipulate YANG data models in a more human-readable format, ala JSON. What lies below digs beneath the higher-level abstractions and should paint a decent picture of the funcitonal nature of Yangson.
Manipulating models with Rosetta and Yangson¶
One of the goals of many network operators is to provide abstractions in a multi-vendor environment. This can be done with YANG and OpenConfig data models, but as they say, the devil is in the details. It occured to me that you should be able to parse configuration from one vendor and translate it to another. Unfortunately as we all know, these configurations don’t always translate well on a 1-to-1 basis. I will demonstrate this process below and show several features of the related libraries along the way.
The following example begins exactly the same as the Cisco parsing tutorial. Let’s load up some Juniper config and parse it into a YANG data model. First, we’ll read the file.
[1]:
from ntc_rosetta import get_driver
import json
junos = get_driver("junos", "openconfig")
junos_driver = junos()
# Strip any rpc tags before and after `<configuration>...</configuration>`
with open("data/junos/dev_conf.xml", "r") as fp:
config = fp.read()
print(config)
<configuration>
<interfaces>
<interface>
<name>xe-0/0/1</name>
<unit>
<name>0</name>
<family>
<ethernet-switching>
<interface-mode>access</interface-mode>
<vlan>
<members>10</members>
</vlan>
</ethernet-switching>
</family>
</unit>
</interface>
<interface>
<name>xe-0/0/3</name>
<unit>
<name>0</name>
<family>
<ethernet-switching>
<interface-mode>trunk</interface-mode>
<vlan>
<members>10</members>
<members>20</members>
</vlan>
</ethernet-switching>
</family>
</unit>
</interface>
<interface>
<name>xe-0/0/4</name>
<unit>
<name>0</name>
<family>
<ethernet-switching>
<interface-mode>trunk</interface-mode>
<vlan>
<members>VLAN-100</members>
<members>VLAN-200</members>
</vlan>
</ethernet-switching>
</family>
</unit>
</interface>
<interface>
<name>xe-0/0/5</name>
<unit>
<name>0</name>
<family>
<ethernet-switching>
<interface-mode>access</interface-mode>
<vlan>
<members>VLAN-100</members>
</vlan>
</ethernet-switching>
</family>
</unit>
</interface>
</interfaces>
<vlans>
<vlan>
<name>default</name>
<vlan-id>1</vlan-id>
</vlan>
<vlan>
<name>prod</name>
<vlan-id>20</vlan-id>
</vlan>
<vlan inactive="inactive">
<vlan-id>10</vlan-id>
</vlan>
<vlan>
<name>VLAN-100</name>
<vlan-id>100</vlan-id>
</vlan>
<vlan>
<name>VLAN-200</name>
<vlan-id>200</vlan-id>
</vlan>
</vlans>
</configuration>
Junos parsing¶
Now, we parse the config and take a look at the data model.
[158]:
from sys import exc_info
from yangson.exceptions import SemanticError
try:
parsed = junos_driver.parse(
native={"dev_conf": config},
validate=False,
include=[
"/openconfig-interfaces:interfaces",
"/openconfig-network-instance:network-instances/network-instance/name",
"/openconfig-network-instance:network-instances/network-instance/config",
"/openconfig-network-instance:network-instances/network-instance/vlans",
]
)
except SemanticError as e:
print(f"error: {e}")
print(json.dumps(parsed.raw_value(), indent=2))
{
"openconfig-interfaces:interfaces": {
"interface": [
{
"name": "xe-0/0/1",
"config": {
"name": "xe-0/0/1",
"type": "iana-if-type:ethernetCsmacd",
"enabled": true
},
"subinterfaces": {
"subinterface": [
{
"index": 0,
"config": {
"index": 0
}
}
]
},
"openconfig-if-ethernet:ethernet": {
"openconfig-vlan:switched-vlan": {
"config": {
"interface-mode": "ACCESS",
"access-vlan": 10
}
}
}
},
{
"name": "xe-0/0/3",
"config": {
"name": "xe-0/0/3",
"type": "iana-if-type:ethernetCsmacd",
"enabled": true
},
"subinterfaces": {
"subinterface": [
{
"index": 0,
"config": {
"index": 0
}
}
]
},
"openconfig-if-ethernet:ethernet": {
"openconfig-vlan:switched-vlan": {
"config": {
"interface-mode": "TRUNK",
"trunk-vlans": [
10,
20
]
}
}
}
},
{
"name": "xe-0/0/4",
"config": {
"name": "xe-0/0/4",
"type": "iana-if-type:ethernetCsmacd",
"enabled": true
},
"subinterfaces": {
"subinterface": [
{
"index": 0,
"config": {
"index": 0
}
}
]
},
"openconfig-if-ethernet:ethernet": {
"openconfig-vlan:switched-vlan": {
"config": {
"interface-mode": "TRUNK",
"trunk-vlans": [
100,
200
]
}
}
}
},
{
"name": "xe-0/0/5",
"config": {
"name": "xe-0/0/5",
"type": "iana-if-type:ethernetCsmacd",
"enabled": true
},
"subinterfaces": {
"subinterface": [
{
"index": 0,
"config": {
"index": 0
}
}
]
},
"openconfig-if-ethernet:ethernet": {
"openconfig-vlan:switched-vlan": {
"config": {
"interface-mode": "ACCESS",
"access-vlan": 100
}
}
}
}
]
},
"openconfig-network-instance:network-instances": {
"network-instance": [
{
"name": "default",
"config": {
"name": "default"
},
"vlans": {
"vlan": [
{
"vlan-id": 1,
"config": {
"vlan-id": 1,
"name": "default",
"status": "ACTIVE"
}
},
{
"vlan-id": 20,
"config": {
"vlan-id": 20,
"name": "prod",
"status": "ACTIVE"
}
},
{
"vlan-id": 10,
"config": {
"vlan-id": 10,
"status": "SUSPENDED"
}
},
{
"vlan-id": 100,
"config": {
"vlan-id": 100,
"name": "VLAN-100",
"status": "ACTIVE"
}
},
{
"vlan-id": 200,
"config": {
"vlan-id": 200,
"name": "VLAN-200",
"status": "ACTIVE"
}
}
]
}
}
]
}
}
Naive translation¶
Since we have a valid data model, let’s see if Rosetta can translate it as-is.
[159]:
ios = get_driver("ios", "openconfig")
ios_driver = ios()
native = ios_driver.translate(candidate=parsed.raw_value())
print(native)
interface xe-0/0/1
no shutdown
switchport mode access
switchport access vlan 10
exit
!
interface xe-0/0/3
no shutdown
switchport mode trunk
switchport trunk allowed vlan 10,20
exit
!
interface xe-0/0/4
no shutdown
switchport mode trunk
switchport trunk allowed vlan 100,200
exit
!
interface xe-0/0/5
no shutdown
switchport mode access
switchport access vlan 100
exit
!
vlan 1
name default
no shutdown
exit
!
vlan 20
name prod
no shutdown
exit
!
vlan 10
shutdown
exit
!
vlan 100
name VLAN-100
no shutdown
exit
!
vlan 200
name VLAN-200
no shutdown
exit
!
Pretty cool, right?! Rosetta does a great job of parsing and translating, but it is a case of “monkey see, monkey do”. Rosetta doesn’t have any mechanisms to translate interface names, for example. It is up to the operator to perform this sort of manipulation.
Down the Yangson rabbit hole¶
Yangson allows the developer to easily translate between YANG data models and JSON. Most all of these manipulations can be performed on dictionaries in Python and loaded into data models using `from_raw <https://yangson.labs.nic.cz/datamodel.html#yangson.datamodel.DataModel.from_raw>`__. The following examples may appear to be a little obtuse, but the goal is to demontrate the internals of Yangson.
And it’s mostly functional¶
It is critical to read the short description of the zipper interface in the InstanceNode section of the docs. Yanson never manipulates an object, but returns a copy with the manipulated attributes.
Show me the code!¶
Let’s take a look at fixing up the interface names and how we can manipulate data model attributes. To do that, we need to locate the attribute in the tree using the `parse_resource_id <https://yangson.labs.nic.cz/datamodel.html#yangson.datamodel.DataModel.parse_resource_id>`__ method. This method returns an `instance route’. The string passed to the method is an xpath.
[160]:
# Locate the interfaces in the tree. We need to modify this one
# Note that we have to URL-escape the forward slashes per https://tools.ietf.org/html/rfc8040#section-3.5.3
irt = parsed.datamodel.parse_resource_id("openconfig-interfaces:interfaces/interface=xe-0%2F0%2F1")
current_data = parsed.root.goto(irt)
print("Current node configuration: ", json.dumps(current_data.raw_value(), indent=2))
modify_data = current_data.raw_value()
ifname = 'Ethernet0/0/1'
modify_data['name'] = ifname
modify_data['config']['name'] = ifname
stub = current_data.update(modify_data, raw=True)
print("Candidate node configuration: ", json.dumps(stub.raw_value(), indent=2))
Current node configuration: {
"name": "xe-0/0/1",
"config": {
"name": "xe-0/0/1",
"type": "iana-if-type:ethernetCsmacd",
"enabled": true
},
"subinterfaces": {
"subinterface": [
{
"index": 0,
"config": {
"index": 0
}
}
]
},
"openconfig-if-ethernet:ethernet": {
"openconfig-vlan:switched-vlan": {
"config": {
"interface-mode": "ACCESS",
"access-vlan": 10
}
}
}
}
Candidate node configuration: {
"name": "Ethernet0/0/1",
"config": {
"name": "Ethernet0/0/1",
"type": "iana-if-type:ethernetCsmacd",
"enabled": true
},
"subinterfaces": {
"subinterface": [
{
"index": 0,
"config": {
"index": 0
}
}
]
},
"openconfig-if-ethernet:ethernet": {
"openconfig-vlan:switched-vlan": {
"config": {
"interface-mode": "ACCESS",
"access-vlan": 10
}
}
}
}
Instance routes¶
You will notice a goto method on child nodes. You can access successors with this method, but you have to build the path from the root datamodel attribute as seen in the following example. If you aren’t sure where an object is in the tree, you can also rely on its path attribute.
Quick tangent… what is the difference between parse_instance_id and parse_resource_id? The answer can be found in the Yangson glossary and the respective RFC’s.
[161]:
# TL;DR
irt = parsed.datamodel.parse_instance_id('/openconfig-network-instance:network-instances/network-instance[1]/vlans/vlan[3]')
print(parsed.root.goto(irt).raw_value())
irt = parsed.datamodel.parse_resource_id('openconfig-network-instance:network-instances/network-instance=default/vlans/vlan=10')
print(parsed.root.goto(irt).raw_value())
{'vlan-id': 10, 'config': {'vlan-id': 10, 'status': 'SUSPENDED'}}
{'vlan-id': 10, 'config': {'vlan-id': 10, 'status': 'SUSPENDED'}}
What about the rest of the interfaces in the list? Yangson provides an iterator for array nodes.
[162]:
import re
irt = parsed.datamodel.parse_resource_id("openconfig-interfaces:interfaces/interface")
iface_objs = parsed.root.goto(irt)
# Swap the name as required
p, sub = re.compile(r'xe-'), 'Ethernet'
# There are a couple challenges here. First is that Yanson doesn't impliment __len__
# The second problem is that you cannot modify a list in-place, so we're basically
# hacking this to hijack the index of the current element and looking it up from a "clean"
# instance. This is a pet example! It would be much easier using Python dicts.
new_ifaces = None
for iface in iface_objs:
name_irt = parsed.datamodel.parse_instance_id('/name')
cname_irt = parsed.datamodel.parse_instance_id('/config/name')
if new_ifaces:
name = new_ifaces[iface.index].goto(name_irt)
else:
name = iface.goto(name_irt)
name = name.update(p.sub(sub, name.raw_value()), raw=True)
cname = name.up().goto(cname_irt)
cname = cname.update(p.sub(sub, cname.raw_value()), raw=True)
iface = cname.up().up()
new_ifaces = iface.up()
print(json.dumps(new_ifaces.raw_value(), indent=2))
[
{
"subinterfaces": {
"subinterface": [
{
"index": 0,
"config": {
"index": 0
}
}
]
},
"openconfig-if-ethernet:ethernet": {
"openconfig-vlan:switched-vlan": {
"config": {
"interface-mode": "ACCESS",
"access-vlan": 10
}
}
},
"name": "Ethernet0/0/1",
"config": {
"type": "iana-if-type:ethernetCsmacd",
"enabled": true,
"name": "Ethernet0/0/1"
}
},
{
"subinterfaces": {
"subinterface": [
{
"index": 0,
"config": {
"index": 0
}
}
]
},
"openconfig-if-ethernet:ethernet": {
"openconfig-vlan:switched-vlan": {
"config": {
"interface-mode": "TRUNK",
"trunk-vlans": [
10,
20
]
}
}
},
"name": "Ethernet0/0/3",
"config": {
"type": "iana-if-type:ethernetCsmacd",
"enabled": true,
"name": "Ethernet0/0/3"
}
},
{
"subinterfaces": {
"subinterface": [
{
"index": 0,
"config": {
"index": 0
}
}
]
},
"openconfig-if-ethernet:ethernet": {
"openconfig-vlan:switched-vlan": {
"config": {
"interface-mode": "TRUNK",
"trunk-vlans": [
100,
200
]
}
}
},
"name": "Ethernet0/0/4",
"config": {
"type": "iana-if-type:ethernetCsmacd",
"enabled": true,
"name": "Ethernet0/0/4"
}
},
{
"subinterfaces": {
"subinterface": [
{
"index": 0,
"config": {
"index": 0
}
}
]
},
"openconfig-if-ethernet:ethernet": {
"openconfig-vlan:switched-vlan": {
"config": {
"interface-mode": "ACCESS",
"access-vlan": 100
}
}
},
"name": "Ethernet0/0/5",
"config": {
"type": "iana-if-type:ethernetCsmacd",
"enabled": true,
"name": "Ethernet0/0/5"
}
}
]
[163]:
# Translate to Cisco-speak
native = ios_driver.translate(candidate=new_ifaces.top().raw_value())
print(native)
interface Ethernet0/0/1
no shutdown
switchport mode access
switchport access vlan 10
exit
!
interface Ethernet0/0/3
no shutdown
switchport mode trunk
switchport trunk allowed vlan 10,20
exit
!
interface Ethernet0/0/4
no shutdown
switchport mode trunk
switchport trunk allowed vlan 100,200
exit
!
interface Ethernet0/0/5
no shutdown
switchport mode access
switchport access vlan 100
exit
!
vlan 1
name default
no shutdown
exit
!
vlan 20
name prod
no shutdown
exit
!
vlan 10
shutdown
exit
!
vlan 100
name VLAN-100
no shutdown
exit
!
vlan 200
name VLAN-200
no shutdown
exit
!
Hooray! That should work. One final approach, just to show you different ways of doing things. This is another pet example to demonstrate Yangson methods.
[164]:
import re
from typing import Dict
irt = parsed.datamodel.parse_resource_id("openconfig-interfaces:interfaces")
iface_objs = parsed.root.goto(irt)
# Nuke the whole branch!
iface_objs = iface_objs.delete_item("interface")
def build_iface(data: str) -> Dict:
# Example template, this could be anything you like that conforms to the schema
return {
"name": f"{data['name']}",
"config": {
"name": f"{data['name']}",
"description": f"{data['description']}",
"type": "iana-if-type:ethernetCsmacd",
"enabled": True
},
}
iface_data = [
build_iface({
"name": f"TenGigabitEthernet0/{idx}",
"description": f"This is interface TenGigabitEthernet0/{idx}"
}) for idx in range(10, 0, -1)
]
initial = iface_data.pop()
# Start a new interface list
iface_objs = iface_objs.put_member("interface", [initial], raw=True)
cur_obj = iface_objs[0]
# Yangson exposes `next`, `insert_after`, and `insert_before` methods.
# There is no `append`.
while iface_data:
new_obj = cur_obj.insert_after(iface_data.pop(), raw=True)
cur_obj = new_obj
[165]:
# Translate to Cisco-speak
native = ios_driver.translate(candidate=cur_obj.top().raw_value())
print(native)
interface TenGigabitEthernet0/1
description This is interface TenGigabitEthernet0/1
no shutdown
exit
!
interface TenGigabitEthernet0/2
description This is interface TenGigabitEthernet0/2
no shutdown
exit
!
interface TenGigabitEthernet0/3
description This is interface TenGigabitEthernet0/3
no shutdown
exit
!
interface TenGigabitEthernet0/4
description This is interface TenGigabitEthernet0/4
no shutdown
exit
!
interface TenGigabitEthernet0/5
description This is interface TenGigabitEthernet0/5
no shutdown
exit
!
interface TenGigabitEthernet0/6
description This is interface TenGigabitEthernet0/6
no shutdown
exit
!
interface TenGigabitEthernet0/7
description This is interface TenGigabitEthernet0/7
no shutdown
exit
!
interface TenGigabitEthernet0/8
description This is interface TenGigabitEthernet0/8
no shutdown
exit
!
interface TenGigabitEthernet0/9
description This is interface TenGigabitEthernet0/9
no shutdown
exit
!
interface TenGigabitEthernet0/10
description This is interface TenGigabitEthernet0/10
no shutdown
exit
!
vlan 1
name default
no shutdown
exit
!
vlan 20
name prod
no shutdown
exit
!
vlan 10
shutdown
exit
!
vlan 100
name VLAN-100
no shutdown
exit
!
vlan 200
name VLAN-200
no shutdown
exit
!
Deleting individual items¶
Here is an example of deleting an individual item. Navigating the tree can be a bit tricky, but it’s not too bad once you get the hang of it.
[166]:
# Locate a vlan by ID and delete it
irt = parsed.datamodel.parse_resource_id("openconfig-network-instance:network-instances/network-instance=default/vlans/vlan=10")
vlan10 = parsed.root.goto(irt)
vlans = vlan10.up().delete_item(vlan10.index)
print(json.dumps(vlans.raw_value(), indent=2))
[
{
"vlan-id": 1,
"config": {
"vlan-id": 1,
"name": "default",
"status": "ACTIVE"
}
},
{
"vlan-id": 20,
"config": {
"vlan-id": 20,
"name": "prod",
"status": "ACTIVE"
}
},
{
"vlan-id": 100,
"config": {
"vlan-id": 100,
"name": "VLAN-100",
"status": "ACTIVE"
}
},
{
"vlan-id": 200,
"config": {
"vlan-id": 200,
"name": "VLAN-200",
"status": "ACTIVE"
}
}
]