Auto Discovery and Reconciliation

 Configuration Management, Inventory Management  Comments Off on Auto Discovery and Reconciliation
Sep 232013

We have inventory databases (either NIM or CMDB) that we use to manage our network operations. We open up tickets based on this data, we plan our strategies based on this data, we make procurements based on this data.

However, if this data is wrong, we would be loosing money or our quality of service. There will be customers who are not given the correct service, there will be service consuming subscribers who are not on the billing anymore, there will be wrongly configured services or orphan devices that are deployed and forgotten.

The aim of Auto Discovery and Reconciliation tools (which are generally sold as an add-on to the NIM or CMDB) is to collect the real world information and correct any discrepancy between what we see and what we know.

These tools aim to discover network and service topology from the operational devices. The discovery process starts with the input of destination networks (or IPs) and via different types of protocols (SNMP, CORBA, SOAP etc.) they discover the devices. The tools employ “adapters” for different types of devices and protocols. (Here the customers should wisely choose the vendor as some vendors are strong in discovering IT assets while others concentrated more on network assets.)

After the discovery, the data generally are converted into a generalized model. The model could be SID based or propriety.

The second set of adapters would collect data from the master database. That will be the data source where our authorized, real world, operational assets reside. This data should also be converted to the tools model and the reconciliation process can start. (This central standardized model employed by the tools is not a must but a preferred way as the data models that are subject to reconciliation may not be the same)

The discovery and reconciliation is triggered either manually or by a scheduled process. The schedule period may vary from minutes to days. This depends heavily on the nature of the data sources. If the data source changes frequently or tied to revenue this increased the importance of the correctness, therefore smaller intervals are preferred. However for network infrastructure, 1-day interval would be enough.

One important thing to mention is we do not discover the whole network element. In the design phase, we typically decide on the element types (routers, switches, firewalls etc.) and also their attributes (hostname, interfaces etc.) that will be used in the discovery and reconciliation process. What we will discover is heavily dependent on the contents of our main, master data source.

The results of the reconciliation are the deltas between the real world and the “known” world. If there are any deltas, that means our data source which we rely on for our operations is wrong. We should take actions to correct any mistakes. Some actions can be automatically triggered by the tool itself. A naming convention failure on a device can be automatically synced by a provisioning tool. However, some actions would require manual intervention or a human eye. These are for example, unknown devices on the network, unknown interfaces or even unknown customers. In such cases a work order task should automatically assigned to the owner of the element for review and take action. The actions that would be taken by this functional group could be;

– Removal of the element from the network.
– Triggering a corresponding change action that would update the main data source.

To repeat, the ultimate goal is to keep the inventory up-to-date. For an organization whose processes work perfectly the tool should never find any discrepancy. But for non-mature organizations the tool would be a pain in the neck and bring burden on operations. That’s why I always recommend my customers to concentrate on the processes first, than the data as the data is manipulated by the processes.