Network Automation Concepts

Home » Webinars » Network Automation » Network Automation Concepts

This webinar provides an overview of network automation concepts you should master as you progress from easy wins like creating automated reports or device configurations to building automation systems.

Last modified on 2023-11-09 (release notes)


Network Automation Concepts

1:53:10 Data Models in Network Automation

This section introduces data models and helps you answer these questions:

  • How should I structure the data I need to describe devices in my network?
  • Should I describe devices or should I focus on network topology and services?
  • Can I use device-level data to create device configurations while presenting high-level data models to the operators?
  • How could I transform abstracted (high-level) network- and services data models into device-level data models?
  • How could I integrate data model transformation into my automation workflow?
Data Model Introduction 17:55 2021-04-21
Data Model Introduction QA 8:53 2021-04-21
Device-Level Data Models 15:33 2021-04-21
Optimizing a Device-Level Data Model 17:53 2021-04-21
Infrastructure and Services Data Models 11:40 2021-04-21
Data Model Transformations 17:45 2021-04-21
Data Model Transformation in Automation Workflow 23:31 2021-08-28
Slide Deck 3.3M 2021-01-20

Further Reading – Data Models

Network Automation Data Model Optimization
Data Model Transformation Concepts
Data Transformation Example (Jinja2 / Ansible / Makefile)
Complex Data Transformation Example: Lab Topology Building Tool
Source Code for the Lab Topology Building Tool

Further Reading – Data Transformation

Data Model Transformations in Network Automation Solutions
Worth Reading: Data Manipulation in Jinja2

1:20:31 Formatting, Describing, and Storing Data

After you built your data model, you have to create data structures (device, link, network, service, or customer data) in a format readable by humans and machines, describe the data model for documentation and validation purposes, and store the data somewhere.

This section dives deep into presentation formats (XML, JSON, YAML), data description languages (including YANG), and data stores, from text files and Excel to NoSQL databases.

Data Representation 18:56 2021-08-28
Data Model Descriptions 15:32 2021-08-28
YANG and OpenConfig 13:17 2021-08-28
Data Stores 18:56 2021-08-28
Selecting a Data Store 13:50 2021-08-28

Further Reading – Data Representation

Beware XML-to-JSON Information Loss (Junos with Ansible)
XML-to-JSON Information Loss, Cisco Nexus OS Edition
Fixing XML-to-JSON Conversion Challenges

Further Reading – Data Validation

Data Validation with JSON Schema
Simple Data Validation with YANG Using yanglint
Interesting Tool: Schema Enforcer
New Ansible Data Validation Module(s)
Automation Should Prevent Operator Errors
Validating Data in GitOps-Based Automation
Automation Solution: Testing Data Models

Further Reading – Data Stores

Text Files or Relational Database?
Using YAML Instead of Excel in Network Automation Solutions
Growing Beyond Ansible host_vars and group_vars
What’s the Big Deal with Validation?

Other Interesting Blog Posts

What’s the Big Deal with Validation?
What Is Continuous Integration?
Continuous Integration in Network Automation
Firewall Ruleset Automation with CI Pipeline
From Excel to Network Infrastructure as Code with Carl Buchmann

Sample YANG Data Models

YANG Module Classification (RFC 8199)
Service models explained (RFC 8309)
Customer-focused YANG model for L3VPN service delivery
Network-centric L3VPN YANG Model
YANG model for L2VPN service delivery

2:06:53 Testing and Validation

Testing and input validation are extremely important components of any relaible software solution, and you should not skip them while developing a network automation solution.

This section describes the software testing concepts, unit tests, and integration/system tests. Input validation and integration/deployment pipelines will be covered in an upcoming live session in early 2023.

The Importance of Testing and Validation 18:26 2023-02-11

1:15:19 Software Testing

Software Testing Overview 14:00 2023-02-11
Software Testing with Version Control 10:04 2023-02-11
Unit Tests 13:44 2023-02-18
Unit Tests in Network Automation 14:56 2023-02-18
Example - Nexus OS VLAN List 5:33 2023-02-18
Integration and System Tests 17:02 2023-02-18

33:08 Input Data Validation

Input Data Validation 20:18 2023-03-02
Example - VLAN Validation 5:19 2023-03-02
Writing Fail-Safe Playbooks 7:31 2023-03-02

More Information

Podcast: Network Testing with Kristian Larsson and David Barroso
Podcast: Test-Driven Network Development with Michael Kashin
Validating Data in GitOps-Based Automation
Checking Network Device Configurations in a GitOps CI Pipeline
Slide Deck 3.6M 2023-01-18

XML-to-JSON Conversion Details

Beware XML-to-JSON Information Loss (Junos with Ansible)
XML-to-JSON Information Loss, Cisco Nexus OS Edition
Fixing XML-to-JSON Conversion Challenges

51:45 Continuous Integration, Delivery, and Deployment

Once you have a reliable test suite, you can start implementing continuous integration (automated merging of changes to the baseline code branch), delivery (having a deliverable environment at all times) and deployment (automated deployment of the deliverable environment).

This section describes CI/CD principles, their applicability in network automation, and the hidden gotchas of trying to validate network configurations before deploying them.

Continuous Integration Concepts 13:05 2023-03-02
Continuous Integration with Git 13:22 2023-03-02
Continuous Integration in Network Automation 11:13 2023-03-02
Continuous Delivery and Deployment 14:05 2023-03-02
What Is Continuous Integration?
Continuous Integration in Network Automation
What Are You Going to Test in Network Automation CI/CD Pipeline?
If You Have to Simulate Your Whole Network, You're Doing It Wrong
Network Digital Twins Work Best in PowerPoint

37:05 Single Source of Truth

After mastering the data model-, model transformation-, and data store concepts, you're ready for one of the most important network automation topics: single source of truth.

Network Automation Source-of-Truth 17:00 2021-12-17
Building a Source-of-Truth 20:05 2021-12-17
Slide Deck 783K 2021-11-01
Building the Network Automation Source of Truth
Building Network Automation Source-of-Truth (Part 2)
Creating Automation Source-of-Truth from Device Configurations
Device Configurations Are Not a Good Source of Truth
Read Network Device Information with REST API and Store It Into a Database
Building Automation Device Inventory with Open Source Tools
Big Red Button for Network Automation

Further Reading

Git as a Source of Truth for Network Automation
Hierarchical Configuration Up and Running

55:22 Manage Network Device Configurations with Git (GitOps)

One of the first steps on anyone's automation journey should be tight control of device configurations using a version control system. This section describes how you can use Git and GitLab/GitHub to track changes to device configurations, correlate changes to tickets or business requirements, implement review and approval workflow, and finally use Git as the single source of (configuration) truth.

Manage Device Configurations with Git 12:42 2023-11-09
Track Changes to Device Configurations 11:33 2023-11-09
Approve Changes with Merge/Pull Requests 8:37 2023-11-09
Use Feature Branches to Document Changes 6:40 2023-11-09
Change/Approve/Deploy Configurations with Git 15:50 2023-11-09

1:00:36 Network Infrastructure as Code: Concepts

Network Infrastructure as Code (NIAC) is another overhyped buzzword. In this section we'll try to define what NIAC is, why you might care about it, and how to implement simple NIAC in networks that use decent devices with configuration replace functionality.

The second half of the presentation introduces advanced challenges like abstraction layers, dealing with numerous devices and byzantine failures, and using software development methodologies in network automation.

Overview 22:21 2021-12-17
Deployment Process 18:46 2021-12-17
Abstraction Layers 6:52 2021-12-17
Using Software Development Methodologies 12:37 2021-12-17
Slide deck 2.3M 2021-11-01

Blog Posts: Infrastructure-as-Code Concepts

Network Infrastructure as Code Is Nothing New
Infrastructure as Code Actually Makes Sense
Automation Isn’t About Building a Button to Press

Blog Posts: Infrastructure-as-Code Implementations

Adjusting System State with Infrastructure as Code
Infrastructure-as-Code, NETCONF and REST API
Infrastructure-as-Code Tools
From Excel to Network Infrastructure as Code with Carl Buchmann
Network Infrastructure as Database
Can We Make REST API Transactional Across Multiple Calls?
Stop the Low-Level Configuration Manipulation

Further Reading

Guide To GitOps (WeaveWorks)

1:28:48 Intent-Based Networking

Most of the intent-based systems are nothing more than a fancy orchestration system with an abstraction layer. This section describes the many levels of abstraction you can implement in such a system.

What Is Intent-Based Networking 19:00 2022-04-14
Device Configuration as Intent 7:29 2022-04-14
Abstracting and Managing Device-Level Intent 10:18 2022-04-14
Replacing Intent Definition 6:38 2022-04-14
Network-Wide Intent 6:06 2022-04-14
Automated Remediation 12:42 2022-04-14
Back to Reality 16:29 2022-04-14
Questions and Answers 10:06 2022-04-14

Additional resources

Slide deck 2.3M 2022-03-16
Intent-Based Networking-related Blog Posts

Further Reading

Intent-Based Networking Taxonomy
The Maslow’s Pyramid of Needs of Intent-Based Networking by Benoît Claise
How Facebook Learned to Stop Worrying and Love the Network
Circular Dependencies Considered Harmful (World-Wide Facebook Outage)
Automation Solution: Deploy BGP Routing with YANG Data Models
Using Flow Tracking to Build Firewall Rulesets... and the Halting Problem
The Law of Leaky Abstractions (Joel on Software)
The ABC of Vendor Lock-In
Shift Network Spend From Premium Products to Premium People

Service-Oriented IETF Data Models

Service models explained (RFC 8309)
YANG model for L3VPN service delivery
YANG model for L2VPN service delivery

1:48:49 Event-Driven Automation (EDA)

After automating device configurations and service provisioning you might start tackling the holy grail of network automation: changing the network behavior based on real-time external event.

In his March 2018 presentation, David Gee described the fundamentals of Event-Driven Automation (EDA), including:

  • Why would you want event-driven automation and what are its pitfalls?
  • What exactly is an event and what's the difference between signals and events?
  • How would the architecture of an event-driven solution look like?
  • Why do we need event normalization and correlation?

He concluded with an overview of open-source and commercial tools you could use when building an event-driven solution and demonstrated the concepts with two simple examples using StackStorm and Salt.

Introduction to Event-Driven Network Automation 25:26 2023-10-04
What Is an Event? 15:44 2023-10-04
Event Normalization and Correlation 29:10 2023-10-04
Event-Driven Automation Solutions 15:44 2023-10-04
Demonstrations 22:45 2023-10-04
Slide Deck: Event-Driven Automation 7.9M 2023-10-04
You started this section on %started% Mark completed