Device backup YAML: ignore patterns, diff ignore, and secret redaction

Netpicker uses platform-specific YAML files to describe how a device backup should be collected and cleaned up. Three settings are especially useful when backup output contains noisy lines or sensitive values:

• ignore_patterns removes matching lines before the backup file is saved.
• diff-ignore hides matching lines when Netpicker reads a saved backup for normal comparison or display.
• redact-secrets replaces secrets before the backup is stored.

They sound similar, but they are meant for different jobs. This guide explains the difference and shows where to place each option in a backup YAML file.

Where these settings live

Backup YAML files are structured around one or more protocols. A simplified file looks like this:

protocols:
  default:
    transfers:
      screen:
        - show-run: show running-config
          ignore_patterns:
            - Building configuration

    diff-ignore:
      - Last configuration change

    redact-secrets:
      case-sensitive: true
      patterns:
        - snmp-server community (\S+)
        - BANNER: banner \w+ \^C.*?\^C

The important placement rule is:

• ignore_patterns belongs on a screen transfer command.
• diff-ignore belongs on the protocol, for example under protocols.default.
• redact-secrets also belongs on the protocol.

1. ignore_patterns: remove lines before storing the backup

Use ignore_patterns when the command output itself contains lines that should not be written into the saved configuration file.

Typical examples are banner-like command output lines such as:

Building configuration...
Config generation may take some time ...
Press Enter to continue...

These lines are not really part of the configuration. If they are saved, they can create confusing backup content. With ignore_patterns, Netpicker skips matching lines while writing the command output to the backup file.

protocols:
  default:
    transfers:
      screen:
        - show-run: show running-config
          kwargs:
            read_timeout: 30
          ignore_patterns:
            - Building configuration

Each entry is a regular expression. If a stripped output line matches one of the expressions, that line is not written to the backup file.

Use ignore_patterns for lines you never want in the stored backup when using a screen-based backup command.

2. diff-ignore: hide noisy lines during comparison

Use diff-ignore when a line should remain in the raw stored backup, but should not affect normal backup comparison or display.

This is useful for values that change often but are not meaningful configuration changes, for example timestamps or encrypted values that are regenerated by the device.

protocols:
  default:
    transfers:
      screen:
        - show-run: show running-config

    diff-ignore:
      - Last configuration change

Fortinet-style output is another good example:

protocols:
  default:
    transfers:
      screen:
        - show-run: show full-configuration

    diff-ignore:
      - ENC
      - set private-key "[^"]+"

Netpicker stores the full backup, then applies these patterns when reading the configuration for normal, non-raw output. Matching lines are filtered out of that view, so they do not create noisy differences.

Use diff-ignore for lines that are useful to keep in the raw backup, but should not create diff noise.

3. redact-secrets: remove sensitive values before storing

Use redact-secrets for passwords, keys, SNMP communities, certificates, pre-shared keys, and other sensitive values that should not be stored in clear text.

protocols:
  default:
    transfers:
      screen:
        - show-run: show running-config

    redact-secrets:
      case-sensitive: true
      patterns:
        - X509: crypto pki certificate .*?quit
        - BANNER: banner \w+ \^C.*?\^C
        - snmp-server community (\S+)
        - (?:password|secret|key-string) (?:[0-9] )?(\S+)

Redaction is applied after the backup is collected and before the final backup file is stored. That means the stored backup contains the redacted output, not the original secret.

Pattern formats

The patterns list supports a few readable formats:

redact-secrets:
  patterns:
    # Plain regex. Netpicker labels it automatically as pattern #0, pattern #1, and so on.
    - snmp-server community (\S+)

    # Label plus regex.
    - WLAN_PSK: config wlan security wpa akm psk (?:set|hex) (\S+)

    # Explicit object form.
    - label: BANNER
      pattern: banner \w+ \^C.*?\^C

Labels make the redacted output easier to understand. For example, a captured SNMP community may become:

snmp-server community <redacted SNMP_COMM>

Capture groups matter

If a redaction regex contains one or more capture groups, only the captured values are replaced. This is usually what you want for single-line secrets:

- snmp-server community (\S+)

Input:

snmp-server community public RO

Output:

snmp-server community <redacted pattern #0> RO

If a regex has no capture group, the entire match is replaced by a redaction marker. This is useful for blocks such as certificates or banners:

- X509: crypto pki certificate .*?quit

By default, the replacement marker uses ! as the line comment prefix:

! ... [REDACTED X509] ...

You can change that prefix:

redact-secrets:
  line-comment: "#"
  patterns:
    - CERT_BLOCK: BEGIN CERTIFICATE.*?END CERTIFICATE

Case sensitivity

Redaction is case-insensitive by default. Set case-sensitive: true if the device syntax should be matched exactly:

redact-secrets:
  case-sensitive: true
  patterns:
    - (?:password|secret|key-string) (?:[0-9] )?(\S+)

Which one should you use?

• Use ignore_patterns when the line is not configuration and should never be saved.
• Use diff-ignore when the line may be useful in the raw backup, but should not create comparison noise.
• Use redact-secrets when the value is sensitive and should not be stored in clear text.

Recommended setup workflow

1. Start with a working backup YAML file for the platform.
2. Run a backup and inspect the saved configuration.
3. Add ignore_patterns for non-configuration lines produced by the backup command.
4. Add diff-ignore for timestamps, generated encrypted values, or other noisy lines that should remain available in raw output.
5. Add redact-secrets for secrets, keys, communities, passwords, certificates, and private key material.
6. Run another backup and verify both the stored raw backup and the normal diff view.

Complete example

protocols:
  default:
    transfers:
      screen:
        - show-run: show running-config
          kwargs:
            read_timeout: 60
          ignore_patterns:
            - Building configuration
            - Current configuration.*bytes

    diff-ignore:
      - Last configuration change
      - NVRAM config last updated

    redact-secrets:
      case-sensitive: true
      line-comment: "!"
      patterns:
        - SNMP_COMMUNITY: snmp-server community (\S+)
        - LOCAL_USER_SECRET: username \S+ secret (?:[0-9] )?(\S+)
        - PASSWORD_LINE: (?:password|secret|key-string) (?:[0-9] )?(\S+)
        - CERTIFICATE_BLOCK: crypto pki certificate .*?quit

This setup keeps command noise out of the saved backup, prevents timestamp-style lines from creating diff noise, and ensures sensitive values are redacted before the backup is stored.

Practical tips

• Keep patterns as specific as possible. Broad expressions can hide real configuration changes.
• Use capture groups for single secret values, so the surrounding configuration line stays readable.
• Use labelled patterns for anything important. Labels make redacted backups easier to audit.
• Test with real backup output before rolling a new YAML change into production.
• If a value is sensitive, prefer redact-secrets over diff-ignore. Diff filtering does not remove the value from the stored raw backup.