Log Data Outranks Traditional Data Sources for Network Operations Management

Key phrase:: Network Operations – and that should be no surprise.

While the network is a important component of an APM initiative, it is usually the last area to participate collaboratively. (I’ve seen things!) And logs are always part of the APM toolset. But logs alone only help a few folks, leaving nothing for the application stakeholders.

Successful APM is achieved when all of the ‘traditional IT silos’ are contributing and participating – network, security, middleware, mainframe, operations. APM focuses on transaction and application characteristics but you cannot get to “root cause” unless the other silos participate – otherwise, full-on “blame game”. Getting those silos to collaborate – it’s all about the process – and not the technology – as this article reminds us.

So if your network group lives on logs – that’s cool. Can they baseline performance? Do they share the KPIs? Do they proactively notify when things ‘go south’? Do they post configuration changes so that you can correlate your transaction degradations with ongoing network issues?

Building data landfills, based on logs, and consumed by a few specialists – I don’t get the benefit. But reduce the data to KPIs and baselines, update proactively and contribute to the single view of end-2-end service performance… that’s what I would put some budget chips on! 😉

From what angle should we be looking at APM as it relates to IT strategy? | response only

How about the angle of “value to the business”? It doesn’t matter if you are monitoring infrastructure, real-time performance or end-user experience – or all three – if the business is not deriving a value from that investment. DevOps especially is really about software/service quality. Monitoring is just a means of getting there.

So does infrastructure monitoring help me improve software/service quality? Nope. Just lets me measure the size (and frequency) of the crater. A lot of folks cluster applications not to get performance but because they can’t fix the memory leaks or other instabilities. So the gap is in getting code fixed – not measuring the craters!

So does real-time performance monitoring improve software/service quality? Nope. I get to see what is breaking, in great detail. But I don’t have a business-impact context to let me prioritize where I should be spending my spare software-fix budget. So I throw a few more instances into the mix… and hope I meet my SLA.

So does end-user-experience improve software quality? Nope. I finally get to see the business context, specifically which transactions and what business impact – but I don’t get much of a clue to what is broken – other than it is transaction #47… and it’s costing us a ton!

To make sense, APM needs all of the info. So let’s throw in network traffic monitoring (different from infrastructure) and logs (for those apps we can’t instrument or are using RPCs, whatever). And hopefully I’ll have visibility into what needs to get fixed. But then you have to have processes on which to employ all these measurement, and when you catch problems, you have a mechanism to correct them. And this is something best done before you even hit production – but that’s a lesson for another day.

APM – Application Performance Management – is a very well established technology (~20 years old) for gathering and disseminating performance information about business applications.  This simple concept generates a lot of complexity because of the very broad impact it can have on an organization’s IT Operations, the Business Sponsor, Software Testing and Software Development.  There are also some complexities introduced by vendor positioning – claims, features, functionality and capabilities.  And there is also a fluid set of priorities and terminology introduced by the various research firms who evaluate the vendors and define the marketplace.  To the newcomer – it’s a mess of conflicting messages.  To the ‘salty dog’ – it’s just software sales and marketing 101.

My goal is to cut through the hype and focus on how you actually live with and employ these tools.  What do you really get?  What do you need to do to be successful?  What are the impediments to success you need to avoid?  What should you be doing to be a superstar?  Most of this will be from the ‘practitioner viewpoint’ – how exactly how I get it done with the artifacts I’ll make available under open-source principles.  And I will also discuss the tools I’m looking for and what I think the future may bring.  I don’t want the recipe for success to be a secret any longer nor overburdened by the unavoidable market positioning.

I’ve spent the last 12 years collecting and refining best practices around performance management but was unable to establish a consistent delivery model for bringing these techniques to the organizations that need them.  That’s on me – I could not get the elephant to dance.  Part of it was because the original offerings were considered proprietary as they conferred a powerful competitive advantage in selling performance management software.  No one likes to give that advantage away.  At the same time, the techniques were actually “vendor neutral” and could exploit any variety of tools in establishing a capable practice around APM, not simply the product I was representing at the time.  This makes for an uncomfortable feeling among the sales team as the product distinctions dissolve away, even as it would result in stunning deal-size multiples later.  When you educate a customer on how to be successful with technology, they never forget who gave them that education.

But the biggest impediment is that establishing a viable, renewable, APM practice simply cannot be achieved in a calendar week or two.  While it actually only needs about 100-150 hrs, these contact hours need to be delivered over 6-18 months.  This is what I call discontinuous service delivery and it makes consulting/services organizations cringe!  It makes it really hard to schedule resources to be on-site.  It makes it really hard to keep consultants busy and billable (utilization target and bonus).  It makes it really hard to predict revenue and long-term staffing needs.  Over the years, I’ve spoken with dozens of integrator and service firms and this is the absolute reality – they simply cannot solve the ask within the traditional services business model.

Why does it take 6-18 months when the technology can be deployed in days!!!

Simple.  In my experience, 95% of achieving a successful APM practice is about organizational change.  It really has almost nothing to do with the tools and technology.  Really.  The tools generate the data easy enough but harnessing the data – that’s where the bulk of organizational change is required.  This APM visibility is something that folks have not had before, nor have they had to consider bringing all of the stakeholders in coordination in employing the data.

So what is actually happening, over the 6-18 months that it takes an APM practice to be successfully established, is process re-engineering, process documentation, acceptance criteria AND a deployment phase of some 12-16 weeks, depending on scope.

So what processes?  What is a phased deployment?  Who staffs this initiative?  What does my project manager do?  How do I get funding?  Show me a schedule!  When can we start!!

These are the questions that I will be answering, over the next couple months.  I’m planning a bunch of videos – should be cool.  If you really, really can’t wait, I already told the story (spoiler alert) in APM best practices: Realizing Application Performance Management – check it out (paper and electronic).  I’ve also collected some other papers, articles PPT and webinars here, and I’ll be adding to this resource page as I find (or contribute) items of merit.

One of the upcoming videos will be a discussion of the service offerings available from The APM Practice, LLC.  I’m using a retainer-based model to solve the discontinuous service delivery problem, not unlike a legal practice.  You can get access to my experience and wisdom, as needed, throughout your journey in harnessing APM visibility, remote and on-site, as appropriate.

But I really expect most of you all, to do this yourselves.  So the videos will include the same introductory presentation I make to prospective clients.  You’ll see the same proposal overview for a Service Bureau or COE.  You’ll see the same introduction to the APM Maturity Model as well as the typical technology topics I am asked to illuminate.  As APM vendors allow, I’ll even share the mentoring sessions I deliver that demonstrate exactly how to employ a given tool to participate in a specific process – but this is a stretch goal.  The primary goal of this consultancy is to provide the materials so that a couple of hours on your part will be useful to guide and accelerate your own initiative.  And if it is too hard a problem – then that’s where I can come in and join your team.

Another feature I’ll be implementing is to establish a practitioner track.  There are a variety of roles that an APM initiative will introduce.  The APM Specialist role is a particular track that was used to mentor candidates for the APM SWAT team that I recently left.  This is a self-paced mentoring program, not unlike the Data Scientist track at Coursera.  The mentoring program was pretty effective at going from ‘zero to hero’ for internal candidates.  There is no reason some of you might not benefit from a little more structure in your own practice.

Could I also offer an “APM Practitioner Certification”?  Might be a good idea – your comments will let me know.  The Skills Assessment practice component was originally established to evaluate reseller progress with APM.  The Competency exercises in the book are used during a Service Bureau or COE project, to ensure folks demonstrate the appropriate skills set.  All the foundation is ready.

Could I also put the whole thing on Coursera?  I’d be open to that but we all need to keep perspective – APM is really a highly specialized community – not a lot of people!  Let’s see what we can accomplish through our APM Community.  But if the demand is there – why not! My commitment is to ‘tear down the wall’ about how to be successful with APM.  There is no future success for me – unless there is ongoing success for you.  Believe me – I am “all in” on this one!

Cheers.