ServiceNow Discovery: MID Server Architecture
Introduction
A sound MID Server Architecture is critical to the success of your ServiceNow Discovery implementation. I have been involved in too many MID Server deployments and I plead guilty of initially, not having a simple point of view for what the MID Server Architecture should look like.
Even before I started working at QuickNexus, I knew Customers paid big-bucks to my employers for my recommendations.
And, unfortunately, I took the complexity of the solution as a measure of my expertise, my indispensability and the quality of my recommendation. But, now I know how wrong I was.
Every customer has something unique about their infrastructure and because of that, all the focus starts there. We solve for the exception and then try to accommodate the standardized infrastructure as an afterthought. As a result, we start with a complex structure that we are stuck with and are unable to get out of, since the foundation is overcomplicated.
So, I took a step back and perused numerous projects I was a part of, to study similarities. What I came back with, is an approach to start with the simplest architecture first and accommodate for exceptions, if applicable, at the end.
And to my surprise, with the foundation of simplicity, the exceptions initially identified, made an exception to themselves and were accommodated in the simple MID Server Architecture. I have captured some of these observations later in the article.
I put down the following guidelines, that I personally use to approach a MID Server Architecture.
And, to remind myself to follow my own advice, I wrote the heading on the next section… which is really directed to no one else, but me…
Simplicity is cool. If its complex, you are a fool!
It is really easy to be build and be proud of a complicated MID Server deployment strategy. And then, when you have to maintain it, you secretly hate yourself for building it that way.
Maybe you think simplicity undermines you as an expert or maybe its just job security knowing that only you can make sense of how all the MID Servers work together! Or maybe, you don’t know better, until now…
If you can not summarize your MID Server Architecture in one sentence, it is too complicated.
A complicated architecture that does not fit in the templates below results in your organization spending more resources on maintaining MID Servers that it should.
While the recommendations below are for ServiceNow Discovery, they can be intuitively extended to other MID Server use-cases.
MID Server Architecture
The recommendations below are aligned with the Prod Instance. The MID Server Design exercise, which should follow, details how to handle MID Servers for non-Prod. Instances.
For starters, the MID Servers should be located on your network that has the maximum visibility to the rest of the network. For example, if your Networks are segmented, put MID Servers in the Prod Network, as it typically has access to the non-Prod Network (and it’s typically not the other way).
If your organization has a flat network, good for you! Read on…
Approach the recommendations below from top to bottom, moving to the next, ONLY if the previous one does not work due to performance limitations or security restrictions.
The Cluster in parenthesis - beside each section - is to allow for more than one MID Server in the architecture to accommodate large deployments. Plus, you should always use MID Server Clusters.
1: One MID Server (Cluster)
There is nothing better than having a single MID Server to maintain at a single location. For performance and availability reasons, you may have two or more MID Servers in a Cluster.
Apart from the sheer ease of maintenance, it also simplifies the network firewall configurations and maintenance required to make sure that the MID Servers can actually reach various segments on the network. For this reason, the location of the MID Servers should be strategically selected so that we can maximize the network visibility for the MID Server.
While, most may think this is too simple to implement in their organization due to network segmentation, I have experienced something different. What I have often seen is that devices in these network segments, are for the same reason they are segmented, typically out of Discovery scope.
We have implemented this simple architecture for several customers. Coupling MID Servers and Schedules, supported with a rationalized Schedule Design has resulted in amazing outcomes, especially in cases where we simplified the MID Server Architecture by 10x!
The biggest challenge we faced while deploying this architecture is to convince the customer (and even ourselves) that simplicity is indeed superior to complexity.
2: One MID Server (Cluster) per Country / Geography
This builds on the single MID Server Concept above and extends it to Global Customers as latency plays a significant role in the throughput of Discovery.
It’s often prudent to move the MID Servers closer to their targets to minimize latency and accommodate for scheduling aligned with local time-zones.
3: One MID Server (Cluster) for Data Centers and One MID Server (Cluster) for Office Locations
Segregating MID Servers for Data Centers vs Office Locations is primarily to prevent overlap in the Discovery Schedules. This applies even more if you are Discovering all your Data Centers using one MID Server spanning different time zones.
Additionally, Office Locations are often on different segments of the network, with restricted access to the Data Center Network. And if we intend to discover end-user devices at Office Locations, we often run Discovery Schedules multiple times during the day, without any impact to the consistency we require from Data Center discovery.
4: One MID Server (Cluster) for Data Centers and One MID Server (Cluster) for Office Locations per Country / Geography
This builds on the single MID Server Concept above and extends it to Global Customers as latency plays a significant role in the throughput of Discovery.
It’s often prudent to move the MID Servers closer to their targets to minimize latency and accommodate for scheduling aligned with local time-zones.
5: One MID Server (Cluster) per Data Center Location and One MID Server (Cluster) per Office Location
This supports the highest level of granularity for some of the most complex environments I have worked with, while allowing for sanity in how the numerous MID Servers are deployed.
While the number of MID Servers with this approach may be more than with a more complex architecture, the simplicity of this approach allows for ease of maintenance and the ability to accommodate changes to the infrastructure without yet another MID Server Architecture discussion.
It also minimizes firewall exceptions and proves to be extremely agile for large, complex and ever-changing landscapes.
X: Exceptions
All of us get a negative point for every MID Server we put in this category.
Here are several ways to question these exceptions and try to incorporate them into one of the recommended design templates above:
Are the CIs in these network segments in Discovery scope?
Are our assumptions for these infrastructure exceptions accurate?
Can we make approved network changes to the infrastructure to eliminate the exception?
And, yes, if you still end up having a small subset of exceptions, its ok. While exceptions are a necessary and often entertained due to the value they bring to operations, we have to be careful so that they do not become the norm.
Conclusion
If you are reading this article before your first MID Server deployment, lucky you! You have just saved your organization tons of resources and paved your way to a reliable foundation to run ServiceNow Discovery.
As for others, who already have a complicated MID Server deployment, our recommendation is to build this out in parallel and switch to a simple architecture, before retiring your existing MID Servers.
Based on multiple exercises in successfully helping customers transition to a simpler architecture, I also recommend you request new MID Server Hosts, even though it may result in duplicated infrastructure in the interim. This supports a seamless transition to a new MID Server Architecture.
I welcome comments and inputs to my recommendations above.
