PySMI: developer documentation¶

Running MIB transformation¶

MibCompiler class instance should be used for MIB module transformation.

class pysmi.compiler.MibCompiler(parser, codegen, writer)¶

Top-level, user-facing, composite MIB compiler object.

MibCompiler implements high-level MIB transformation processing logic. It executes its actions by calling the following specialized objects:

• readers - to acquire ASN.1 MIB data
• searchers - to see if transformed MIB already exists and no processing is necessary
• parser - to parse ASN.1 MIB into AST
• code generator - to perform actual MIB transformation
• borrowers - to fetch pre-transformed MIB if transformation is impossible
• writer - to store transformed MIB data

Required components must be passed to MibCompiler on instantiation. Those components are: parser, codegenerator and writer.

Optional components could be set or modified at later phases of MibCompiler life. Unlike singular, required components, optional one can be present in sequences to address many possible sources of data. They are readers, searchers and borrowers.

Examples:

from pysmi.reader.localfile import FileReader
from pysmi.searcher.pyfile import PyFileSearcher
from pysmi.searcher.pypackage import PyPackageSearcher
from pysmi.searcher.stub import StubSearcher
from pysmi.writer.pyfile import PyFileWriter
from pysmi.parser.smi import SmiV2Parser
from pysmi.codegen.pysnmp import PySnmpCodeGen, baseMibs

mibCompiler = MibCompiler(SmiV2Parser(),
                          PySnmpCodeGen(),
                          PyFileWriter('/tmp/pysnmp/mibs'))

mibCompiler.addSources(FileReader('/usr/share/snmp/mibs'))

mibCompiler.addSearchers(PyFileSearcher('/tmp/pysnmp/mibs'))
mibCompiler.addSearchers(PyPackageSearcher('pysnmp.mibs'))

mibCompiler.addSearchers(StubSearcher(*baseMibs))

results = mibCompiler.compile('IF-MIB', 'IP-MIB')

Creates an instance of MibCompiler class.

Parameters:	parser – ASN.1 MIB parser object codegen – MIB transformation object writer – transformed MIB storing object

addBorrowers(*borrowers)¶

Add more transformed MIBs repositories to borrow MIBs from.

Whenever MibCompiler.compile encounters MIB module which neither of the searchers can find or fetched ASN.1 MIB module can not be parsed (due to syntax errors), these borrowers objects will be invoked in order of their addition asking each if already transformed MIB can be fetched (borrowed).

Parameters:	borrowers – borrower object(s)
Returns:	reference to itself (can be used for call chaining)

addSearchers(*searchers)¶

Add more transformed MIBs repositories.

MibCompiler.compile will invoke each of configured searcher objects in order of their addition asking each if already transformed MIB module already exists and is more recent than specified.

Parameters:	searchers – searcher object(s)
Returns:	reference to itself (can be used for call chaining)

addSources(*sources)¶

Add more ASN.1 MIB source repositories.

MibCompiler.compile will invoke each of configured source objects in order of their addition asking each to fetch MIB module specified by name.

Parameters:	sources – reader object(s)
Returns:	reference to itself (can be used for call chaining)

compile(*mibnames, **options)¶

Transform requested and possibly referred MIBs.

The compile method should be invoked when MibCompiler object is operational meaning at least sources are specified.

Once called with a MIB module name, compile will:

• fetch ASN.1 MIB module with given name by calling sources
• make sure no such transformed MIB already exists (with searchers)
• parse ASN.1 MIB text with parser
• perform actual MIB transformation into target format with code generator
• may attempt to borrow pre-transformed MIB through borrowers
• write transformed MIB through writer

The above sequence will be performed for each MIB name given in mibnames and may be performed for all MIBs referred to from MIBs being processed.

Parameters:	mibnames – list of ASN.1 MIBs names options – options that affect the way PySMI components work
Returns:	A dictionary of MIB module names processed (keys) and MibStatus class instances (values)

MIB Transformation results¶

MibStatus class instance is used by MibCompiler.compiler() to indicate the outcome of MIB transformation operation.

class pysmi.compiler.MibStatus¶

Indicate MIB transformation result.

MibStatus is a subclass of Python string type. Some additional attributes may be set to indicate the details.

The following MibStatus class instances are defined:

• compiled - MIB is successfully transformed
• untouched - fresh transformed version of this MIB already exisits
• failed - MIB transformation failed. error attribute carries details.
• unprocessed - MIB transformation required but waived for some reason
• missing - ASN.1 MIB source can’t be found
• borrowed - MIB transformation failed but pre-transformed version was used

Fetching ASN.1 MIBs¶

PySMI offers three distinct transport mechanisms for fetching MIBs by name from specific locations. In all cases MIB module name to file name match may not be exact – some name fuzzying can be performed to mitigate possible changes to MIB file name.

FileReader class instance looks up MIB files in given directories on the host running PySMI.

class pysmi.reader.localfile.FileReader(path, recursive=True, ignoreErrors=True)¶

Fetch ASN.1 MIB text by name from local file.

FileReader class instance tries to locate ASN.1 MIB files by name, fetch and return their contents to caller.

Create an instance of FileReader serving a directory.

Parameters:	path (str) – directory to search MIB files
Keyword Arguments:
	recursive (bool) – whether to include subdirectories ignoreErrors (bool) – ignore filesystem access errors

HttpReader class instance tries to download MIB files using configured URL.

class pysmi.reader.httpclient.HttpReader(host, port, locationTemplate, timeout=5, ssl=False)¶

Fetch ASN.1 MIB text by name from a web site.

HttpReader class instance tries to download ASN.1 MIB files by name and return their contents to caller.

Create an instance of HttpReader bound to specific URL.

Parameters:	host (str) – domain name or IP address of web server port (int) – TCP port web server is listening locationTemplate (str) – location part of the URL containing @mib@ magic placeholder to be replaced with MIB name fetch.
Keyword Arguments:
	timeout (int) – response timeout ssl (bool) – access HTTPS web site

FtpReader class instance tries to download MIB files form configured FTP server.

class pysmi.reader.ftpclient.FtpReader(host, locationTemplate, timeout=5, ssl=False, port=21, user='anonymous', password='anonymous@')¶

Fetch ASN.1 MIB text by name from FTP server. FtpReader class instance tries to download ASN.1 MIB files by name and return their contents to caller.

Create an instance of FtpReader bound to specific FTP server directory.

Parameters:	host (str) – domain name or IP address of web server locationTemplate (str) – location part of the directory containing @mib@ magic placeholder to be replaced with MIB name fetch.
Keyword Arguments:
	timeout (int) – response timeout ssl (bool) – access HTTPS web site port (int) – TCP port web server is listening user (str) – username at FTP server password (str) – password for username at FTP server

Blocking unwanted transformations¶

There are cases when MIB transformation may or must not be performed. Such cases include:

• foundation MIBs containing manually implemented pieces or ASN.1 MACRO’s
• obsolete MIBs fully reimplemented within modern MIBs
• already transformed MIBs

MibCompiler expects user to supply a searcher object that would approve or skip MIB transformation for particular name based on whatever reason it is aware of.

In general, searcher logic is specific to target format. Here we will speak solely of pysnmp code generation backend.

Transformed MIBs that were saved in form of Python files can be checked with PyFileSearcher class instances.

class pysmi.searcher.pyfile.PyFileSearcher(path)¶

Figures out if given Python file (source or bytecode) exists at given location.

Create an instance of PyFileSearcher bound to specific directory.

Parameters:	path (str) – path to local directory

Some MIBs, most frequently the base ones, can be stored at a Python package. There existence can be checked with the PyPackageSearcher class instances.

class pysmi.searcher.pypackage.PyPackageSearcher(package)¶

Figures out if given Python module (source or bytecode) exists in given Python package.

Python package must be importable.

Create an instance of PyPackageSearcher bound to specific Python package.

Parameters:	package (str) – name of the Python package to look up Python modules at.

Foundation or obsolete MIBs that should never be transformed can be blindly excluded by listing their names at the StubSearcher class instance.

class pysmi.searcher.stub.StubSearcher(*mibnames)¶

Figures out if given MIB module is present in a fixed list of modules.

Create an instance of StubSearcher initialized with a fixed list or MIB modules names.

Parameters:	mibnames (str) – blacklisted MIB names

A pysnmp-specific list of MIB names to be permanently excluded from transformation can be found at pysmi.codegen.pysnmp.baseMibs.

Tackling MIB parser¶

SNMP MIBs are written in two kinds of special language - SMIv1 and SMIv2. The first SMI version is obsolete, most MIBs by now are written in SMIv2 grammar. There are also efforts aimed at improving SMIv2, but those MIBs are in great minority at the time of this writing.

PySMI is designed to handle both SMIv1 and SMIv2. The way it is done is that SMIv2 is considered the most basic and complete, whereas SMIv1 is a specialization of SMIv2 syntax.

For a user to acquire SMIv2 parser the following factory function should be used.

pysmi.parser.smi.parserFactory(**grammarOptions)¶

Factory function producing custom specializations of base SmiV2Parser class.

Keyword Arguments:
	grammarOptions – a list of (bool) typed optional keyword parameters enabling particular set of SMIv2 grammar relaxations.
Returns:	Specialized copy of SmiV2Parser class.

Notes

The following SMIv2 grammar relaxation parameters are defined:

• supportSmiV1Keywords - parses SMIv1 grammar
• supportIndex - tolerates ASN.1 types in INDEX clause
• commaAtTheEndOfImport - tolerates stray comma at the end of IMPORT section
• commaAtTheEndOfSequence - tolerates stray comma at the end of sequence of elements in MIB
• mixOfCommasAndSpaces - tolerate a mix of comma and spaces in MIB enumerations
• uppercaseIdentifier - tolerate uppercased MIB identifiers
• lowcaseIdentifier - tolerate lowercase MIB identifiers
• curlyBracesAroundEnterpriseInTrap - tolerate curly braces around enterprise ID in TRAP MACRO
• noCells - tolerate missing cells (XXX)

Examples:

>>> from pysmi.parser import smi
>>> SmiV1Parser = smi.parserFactory(supportSmiV1Keywords=True, supportIndex=True)

PySMI offers a pre-built collection of parser grammar relaxation options to simplify its use.

from pysmi.parser.dialect import smiV1
from pysmi.parser.smi import parserFactory

SmiV1Parser = parserFactory(**smiV1)

Apparently, many production MIBs were shipped in syntaxically broken condition. PySMI attempts to work around such issues by allowing some extra SMI grammar relaxations. You can enable all those relaxations at once to maximize the number of MIBs, found in the wild, successfully compiled.

from pysmi.parser.dialect import smiV1Relaxed
from pysmi.parser.smi import parserFactory

RelaxedSmiV1Parser = parserFactory(**smiV1Relaxed)

Defining target transformation¶

PySMI is designed to be able to transform ASN.1 MIBs into many different forms. Destination MIB representation is determined by a kind of code generator object being passed to MibCompiler on instantiation.

At the time of this writing, two code generation backends are shipped with PySMI.

class pysmi.codegen.pysnmp.PySnmpCodeGen¶

Builds PySNMP-specific Python code representing MIB module supplied in form of an Abstract Syntax Tree on input.

Instance of this class is supposed to be passed to MibCompiler, the rest is internal to MibCompiler.

class pysmi.codegen.jsondoc.JsonCodeGen¶

Builds JSON document representing MIB module supplied in form of an Abstract Syntax Tree on input.

Instance of this class is supposed to be passed to MibCompiler, the rest is internal to MibCompiler.

class pysmi.codegen.null.NullCodeGen¶

Dummy code generation backend.

Could be used for disabling code generation at MibCompiler.

Borrowing pre-transformed MIBs¶

Some shipped MIBs appear broken beyond automatic repair. To handle such cases PySMI introduces the ‘borrowing’ functionality – MibCompiler ability to go out and take a copy of already transformed MIB from some repository.

This feature is optional so it is user responsibility to configure one or more borrowing objects to MibCompiler.

class pysmi.borrower.pyfile.PyFileBorrower(reader, genTexts=False)¶

Transformed PySNMP MIB modules borrower.

Creates an instance of Borrower class.

Parameters:	reader – a reader object
Keyword Arguments:
	genText – indicates whether this borrower should be looking for transformed MIBs that include human-oriented texts

Storing transformed MIBs¶

Successfully transformed MIB modules’ contents will be passed to writer object given to MibCompiler on instantiation. Current version of PySMI is shipped with the following writer classes.

class pysmi.writer.pyfile.PyFileWriter(path)¶

Stores transformed MIB modules as Python files at specified location.

User is expected to pass PyFileWriter class instance to MibCompiler on instantiation. The rest is internal to MibCompiler.

Creates an instance of PyFileWriter class.

Parameters:	path – writable directory to store Python modules

class pysmi.writer.callback.CallbackWriter(cbFun, cbCtx=None)¶

Invokes user-specified callable and passes transformed MIB module to it.

Note: user callable object signature must be as follows

cbFun(mibname, contents, cbCtx)¶

Creates an instance of CallbackWriter class.

Parameters:	cbFun (callable) – user-supplied callable
Keyword Arguments:
	cbCtx – user-supplied object passed intact to user callback