Checks and Data Comparison Reference¶

This guide provides comprehensive information on configuring checks in the Operational Compliance app, including JMESPath syntax and check type options using jdiff.

Overview¶

The Operational Compliance app uses two key technologies for data processing and comparison:

JMESPath: Used during data collection to extract specific data from command outputs
jdiff: Used during comparison to determine if data matches according to your rules

Understanding when and how each is used will help you configure effective checks.

Part 1: JMESPath - Data Extraction¶

When it's used: During the Collect Check job, when the app runs commands on devices and needs to extract specific data from the output.

Where you configure it: In the Value Path field of a Check Command object.

What it does: Extracts relevant data from complex JSON structures returned by device commands, so you can compare only the data that matters.

JMESPath is a query language for JSON that allows you to extract specific data from complex JSON structures. The Operational Compliance app uses JMESPath to extract relevant data from device command outputs.

Basic JMESPath Expressions¶

Wildcard (`*`)¶

Matches all elements in an array or all keys in an object. This is the most basic form of data extraction - it simply returns all values at the specified level.

// Input data
{
  "interfaces": {
    "GigabitEthernet1/0/1": {"status": "up", "speed": "1000"},
    "GigabitEthernet1/0/2": {"status": "down", "speed": "1000"}
  }
}

// JMESPath: interfaces.*
// Result: [{"status": "up", "speed": "1000"}, {"status": "down", "speed": "1000"}]

Use Case: When you want to extract all interface objects to compare their complete status and configuration.

Array Wildcard (`[*]`)¶

Matches all elements in an array. This is specifically for array processing and is commonly used with TextFSM output which typically returns arrays of objects.

// Input data
{
  "result": [
    {"interface": "GigabitEthernet1/0/1", "status": "up"},
    {"interface": "GigabitEthernet1/0/2", "status": "down"}
  ]
}

// JMESPath: result[*]
// Result: [{"interface": "GigabitEthernet1/0/1", "status": "up"}, {"interface": "GigabitEthernet1/0/2", "status": "down"}]

Use Case: When processing TextFSM output that returns arrays, or when you need to extract all items from a list of network objects.

Key Selection¶

Extract specific keys from objects. This allows you to drill down into nested objects and extract only the fields you need for comparison.

// Input data
{
  "interfaces": {
    "GigabitEthernet1/0/1": {"status": "up", "speed": "1000", "description": "LAN"},
    "GigabitEthernet1/0/2": {"status": "down", "speed": "1000", "description": "WAN"}
  }
}

// JMESPath: interfaces.*.status
// Result: ["up", "down"]

Use Case: When you only want to compare specific fields (like interface status) and ignore other fields (like descriptions or speeds) that might change frequently.

Advanced JMESPath Expressions¶

Projections¶

Create new arrays or objects by transforming data. Projections allow you to reshape data by selecting specific fields and creating new data structures. This is particularly useful for creating comparison-friendly formats.

// Input data
{
  "interfaces": [
    {"name": "GigabitEthernet1/0/1", "status": "up", "speed": "1000"},
    {"name": "GigabitEthernet1/0/2", "status": "down", "speed": "1000"}
  ]
}

// JMESPath: interfaces[*].[name, status]
// Result: [["GigabitEthernet1/0/1", "up"], ["GigabitEthernet1/0/2", "down"]]

What it does: Takes each interface object and extracts only the name and status fields, creating a simplified array of arrays format that's easier to compare.

Use Case: When you want to compare only specific fields across multiple objects, or when you need to create a standardized format for comparison that excludes irrelevant data.

Filters¶

Filter arrays based on conditions. Filters allow you to include only objects that meet specific criteria, reducing the data set to only relevant items.

// Input data
{
  "interfaces": [
    {"name": "GigabitEthernet1/0/1", "status": "up", "speed": "1000"},
    {"name": "GigabitEthernet1/0/2", "status": "down", "speed": "1000"},
    {"name": "Loopback0", "status": "up", "speed": "N/A"}
  ]
}

// JMESPath: interfaces[?status == 'up']
// Result: [{"name": "GigabitEthernet1/0/1", "status": "up", "speed": "1000"}, {"name": "Loopback0", "status": "up", "speed": "N/A"}]

What it does: Uses the ? filter syntax to include only interfaces where the status equals 'up', excluding down interfaces from the comparison.

Use Case: When you only want to monitor active interfaces, or when you need to focus on specific states (like only established BGP peers, only up interfaces, etc.).

Functions¶

Use JMESPath functions to transform data. Functions allow you to perform calculations, aggregations, or transformations on the extracted data.

// Input data
{
  "interfaces": [
    {"name": "GigabitEthernet1/0/1", "status": "up"},
    {"name": "GigabitEthernet1/0/2", "status": "down"}
  ]
}

// JMESPath: length(interfaces[?status == 'up'])
// Result: 1

What it does: Combines filtering and the length() function to count how many interfaces are currently up. First filters for up interfaces, then counts them.

Common Functions:

length(): Count items in an array
sum(): Add up numeric values
avg(): Calculate average of numeric values
max(): Find maximum value
min(): Find minimum value

Use Case: When you need to monitor counts (like number of up interfaces), calculate totals (like total BGP prefixes), or perform other aggregations on your network data.

Reference Key Anchoring¶

The Operational Compliance app extends JMESPath with reference key anchoring using $ symbols. This is crucial for creating meaningful diffs when comparing data.

Basic Reference Key Syntax¶

// Input data
{
  "interfaces": [
    {"name": "GigabitEthernet1/0/1", "status": "up", "speed": "1000"},
    {"name": "GigabitEthernet1/0/2", "status": "down", "speed": "1000"}
  ]
}

// JMESPath: interfaces[*].[$name$, status, speed]
// This tells the comparison engine to use the interface name as the reference key

Complex Reference Key Examples¶

// Input data (BGP peers)
{
  "result": [
    {
      "vrfs": {
        "default": {
          "peerList": [
            {"peerAddress": "10.1.1.1", "state": "Established", "asn": "65001"},
            {"peerAddress": "10.1.1.2", "state": "Idle", "asn": "65002"}
          ]
        }
      }
    }
  ]
}

// JMESPath: result[0].vrfs.default.peerList[*].[$peerAddress$, state]
// This extracts peer state using peer address as reference key

Part 2: jdiff - Data Comparison¶

When it's used: During the Compare Check job, when the app compares two sets of collected data to determine if they match.

Where you configure it: In the Check Type Options field of a Check object.

What it does: Uses the jdiff comparison engine to determine if the extracted data matches according to your specified rules (exact match, tolerance, etc.).

Check Type Options¶

Check type options define how the extracted data should be compared. Each check type supports different options and is passed as kwargs to the underlying jdiff comparison engine.

EXACT_MATCH¶

Performs exact comparison between two datasets.

Basic Usage¶

null

or

{}

Advanced Options¶

{
  "reference_data": "Dict",
  "value_to_compare": "Dict"
}

Use Cases:

Interface status comparison
Device facts comparison
Configuration verification
BGP neighbor state monitoring
LLDP neighbor verification

Real-World Example: Device Facts Comparison¶

This example shows how to compare device facts between two collection runs:

Sample Input Data (from NAPALM get_facts):

href="#__codelineno-11-1">{ "get_facts": { "fqdn": "cat8000v.cisco.com", "model": "C8000V", "uptime": 12180.0, "vendor": "Cisco", "hostname": "cat8000v", "os_version": "Virtual XE Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 17.12.2, RELEASE SOFTWARE (fc2)", "serial_number": "9GPJZGDAAG8", "interface_list": [ "GigabitEthernet1", "GigabitEthernet2", "GigabitEthernet3", "Loopback0", "Loopback10", "Loopback109", "VirtualPortGroup0" ] } }

JMESPath: os_version Check Type: EXACT_MATCH Options: null

What this does: Extracts only the OS version field from the device facts. The comparison will check if the OS version is exactly the same between two collection runs (before and after a change).

Expected Output:

"Virtual XE Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 17.12.2, RELEASE SOFTWARE (fc2)"

TOLERANCE¶

Allows for numerical tolerance when comparing values.

Required Options¶

{
  "tolerance": 10
}

Use Cases:

CPU utilization monitoring (allow 5% tolerance)
Memory usage comparison (allow 100MB tolerance)
Interface counters (allow packet loss tolerance)
BGP prefix count monitoring
Performance metrics comparison

Real-World Example: BGP Prefix Tolerance¶

This example shows how to monitor BGP prefix counts with tolerance for normal fluctuations:

Sample Input Data (from show ip bgp summary):

[
  {"neighbor": "10.1.1.1", "state_pfxrcd": 150, "as": "65001"},
  {"neighbor": "10.1.1.2", "state_pfxrcd": 75, "as": "65002"},
  {"neighbor": "10.1.1.3", "state_pfxrcd": 200, "as": "65003"}
]

JMESPath: [*].[$neighbor$, state_pfxrcd] Check Type: TOLERANCE Options: {"tolerance": 5}

What this does: Extracts BGP neighbor addresses and their received prefix counts. The comparison will allow up to ±5 prefixes difference when comparing prefix counts between two collection runs. This accounts for normal BGP route fluctuations.

Expected Output:

[
  ["10.1.1.1", 150],
  ["10.1.1.2", 75],
  ["10.1.1.3", 200]
]

Examples¶

// Allow 5% tolerance for CPU usage
{"tolerance": 5}

// Allow 100MB tolerance for memory usage
{"tolerance": 100}

// Allow 5 prefix tolerance for BGP routes
{"tolerance": 5}

PARAMETER_MATCH¶

Matches specific parameters against expected values.

Options¶

{
  "params": {
    "os_version": "17.3.1",
    "model": "C9300-48P"
  },
  "mode": "match"
}

Use Cases:

OS version compliance
Hardware model verification
Feature availability checks
Device type validation
Software version enforcement

Real-World Example: OS Version Compliance¶

This example shows how to verify OS version compliance against a target version:

Sample Input Data (from NAPALM get_facts):

{
  "get_facts": {
    "fqdn": "cat8000v.cisco.com",
    "model": "C8000V",
    "vendor": "Cisco",
    "hostname": "cat8000v",
    "os_version": "Virtual XE Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 17.12.2, RELEASE SOFTWARE (fc2)",
    "serial_number": "9GPJZGDAAG8"
  }
}

JMESPath: os_version Check Type: PARAMETER_MATCH Options: {"params": {"os_version": "17.12.2"}, "mode": "match"}

What this does: Extracts the OS version field and checks if it contains the target version "17.12.2". This is useful for compliance checks to ensure devices are running the expected software version.

Expected Output:

"Virtual XE Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 17.12.2, RELEASE SOFTWARE (fc2)"

OPERATOR¶

Uses mathematical operators for comparisons.

Options¶

{
  "params": {
    "operator": "greater_than",
    "operator_data": 100
  }
}

Supported Operators:

greater_than
less_than
greater_than_or_equal
less_than_or_equal
equal
not_equal
in-range (requires operator_data as array [min, max])

Use Cases:

Threshold monitoring
Performance baselines
Resource utilization checks
Optical power level monitoring
Temperature monitoring

Exclude Functionality¶

For EXACT_MATCH checks, you can exclude specific fields from the comparison to reduce noise in the diff output. This is useful when certain fields (like timestamps, counters, or statistics) change frequently but aren't relevant to your compliance check.

Exclude Example¶

This example shows how to exclude noisy fields from comparisons:

Sample Input Data:

{
  "interfaces": {
    "GigabitEthernet1/0/1": {
      "status": "up",
      "speed": "1000",
      "interfaceStatistics": {
        "inBitsRate": 3582.5,
        "inPktsRate": 3.97,
        "outBitsRate": 17327.6,
        "outPktsRate": 2.21
      }
    }
  }
}

JMESPath: * (extract all data) Check Type: EXACT_MATCH Options: null Exclude: ["interfaceStatistics"]

Expected Output (with exclude):

{
  "GigabitEthernet1/0/1": {
    "status": "up",
    "speed": "1000"
  }
}

The interfaceStatistics field is excluded from comparison, focusing only on the operational status and configuration.

REGEX¶

Uses regular expressions for pattern matching.

Options¶

{
  "pattern": "^[0-9]+\\.[0-9]+\\.[0-9]+$"
}

Use Cases:

Version string validation
IP address format verification
Interface name pattern matching

Real-World Example: Version String Validation¶

This example shows how to validate version string format:

Sample Input Data (from show version):

{
  "version": "17.12.2"
}

JMESPath: version Check Type: REGEX Options: {"pattern": "^[0-9]+\\.[0-9]+\\.[0-9]+$"}

What this does: Extracts the version field and validates it matches the expected format (major.minor.patch). This ensures version strings are properly formatted.

Expected Output:

"17.12.2"

Best Practices¶

JMESPath Best Practices¶

Use Reference Keys: Always use $ anchoring for meaningful diffs
Be Specific: Extract only the data you need to compare
Test Expressions: Use JMESPath online tester to validate expressions
Handle Arrays: Use [*] for array processing
Consider Performance: Complex expressions may impact performance

Check Type Best Practices¶

Choose Appropriate Type: Use the simplest check type that meets your needs
Set Realistic Tolerances: Don't set tolerances too high or too low
Validate Options: Ensure JSON syntax is correct
Document Intent: Use descriptive check names and descriptions
Test Thoroughly: Test with various data scenarios

Common Pitfalls¶

Missing Reference Keys: Without $ anchoring, diffs may be meaningless
Incorrect JSON Syntax: Invalid JSON in options will cause errors
Overly Complex Expressions: Keep JMESPath expressions readable
Inappropriate Tolerance: Too high tolerance may hide real issues
Platform Mismatch: Ensure parser and platform are compatible

Testing Your Configuration¶

Before deploying checks in production:

Test JMESPath: Use online JMESPath testers with sample data
Run Sample Collects: Execute collect checks on test devices
Verify Output: Ensure extracted data matches expectations
Test Comparisons: Run compare checks with known differences
Validate Results: Confirm check results are accurate

Resources¶

JMESPath Official Documentation
JMESPath Online Tester
jdiff Documentation (underlying comparison engine)

Checks and Data Comparison Reference¶

Overview¶

Part 1: JMESPath - Data Extraction¶

Basic JMESPath Expressions¶

Wildcard (*)¶

Array Wildcard ([*])¶

Key Selection¶

Advanced JMESPath Expressions¶

Projections¶

Filters¶

Functions¶

Reference Key Anchoring¶

Basic Reference Key Syntax¶

Complex Reference Key Examples¶

Part 2: jdiff - Data Comparison¶

Check Type Options¶

EXACT_MATCH¶

Basic Usage¶

Advanced Options¶

Real-World Example: Device Facts Comparison¶

TOLERANCE¶

Required Options¶

Real-World Example: BGP Prefix Tolerance¶

Examples¶

PARAMETER_MATCH¶

Options¶

Real-World Example: OS Version Compliance¶

OPERATOR¶

Options¶

Exclude Functionality¶

Exclude Example¶

REGEX¶

Options¶

Real-World Example: Version String Validation¶

Best Practices¶

JMESPath Best Practices¶

Check Type Best Practices¶

Common Pitfalls¶

Testing Your Configuration¶

Resources¶

Wildcard (`*`)¶

Array Wildcard (`[*]`)¶