Kari’s Law goes into effect for ALL SYSTEMS nationwide on February 16, 2020, after the expiration of a 2-year grace period allowing system administrators to become compliant. Kari’s Law requires businesses entities with multi-line telephone systems to provide the following capabilities on all phones:
Allow direct dialing to 911, without any prefix
Provide On-Site Notification when 911 is called
Route 911 calls directly to the local 911 PSAP
Avaya Partner Altura Communications has started a public awareness campaign in conjunction with Avaya and 911 Secure that is designed to inform customers of the issues at hand and offer solutions that are both functionally efficient and financially affordable. You can read their 3 part Blog series by Hank Hunt, Mark Fletcher, and Kevin Kito on the topic here:
In part one of our three-part series examining how Kari’s Law came about, guest blogger and Kari’s father, Hank Hunt recalls that terrible night when his life changed forever
In part two of our three-part series examining how Kari’s Law came about, guest blogger Mark Fletcher – ENP, Avaya’s Chief Architect for Worldwide Public Safety Solutions recounts how he becomes involved
9-1-1. It is likely the most well recognized ‘brand’ in the world.
It is the three digits every man, woman and child know by heart recognized around
the world.
To provide a dispatchable location to emergency responders today’s
Enterprise telephone system administrators have relied on the Automatic Number
Identification / Automatic Location Information (ANI/ALI) database for years.
This database provided 911 dispatchers a cross-reference, or reverse lookup of
telephone numbers to critical dispatch location addresses. Another function of
this core component was to work in conjunction with the Selective Router
Database (SRDB) used to route calls in the PSTN to the right agency. This
allowed number portability, among other things
However, it appears CenturyLink has provided their Washington State Enterprise accounts with notice that they plan to discontinue their critical PS-ALI services effective May 8, 2019. After this date, enterprise customers must establish a replacement service of their own through a 3rd party provider.
They specifically state in their customer notice:
“It is the responsibility of the customer to update the PS/ALI database (via a third-party vendor) with the individual station address information.”
A copy of the new Tarifs regarding this change can be found online here.
Time for a change?
This is troubling, as the short window of remediation is going to
require MLTS system operators to make a snap judgment on a technology they may
not fully understand, and the predatory sales tactics of some providers may
hook an Enterprise into a long-term contract for services with a minimal
roadmap to the future of NG911.
For our valued Avaya customers, WE CAN HELP. Don’t get caught up
in the alphabet soup of acronyms you barely understand. We have our customer’s
best interests at heart, and there are many ways to address this problem. Our
team of industry experts is standing by ready to help you assess your current
position, and guide new solutions that solve the p[roblem today, and well into
the future. Kari’s Law is federally mandated starting February 16, next year,
and the Ray Baum’s Act Section 506, is following close behind requiring
dispatchable addresses.
The Avaya SENTRY™ solution was designed and built to provide full
NG911 functionality, even over today’s legacy networks, complete compliance
with all laws (active now and in the future), and deliver critical pre-arrival
instructions to 1st responders with the actionable information they need to do
their jobs. We are here for you to help you decide what’s right for YOUR
business.
It is no great secret that I am a total advocate for technology related advances for our nation’s Public Safety professionals. When it comes to the FIRST 1st Responders that provide the initial level of security for the public, the need for advancement could not be more relevant. Regardless of what you call these dedicated folks; Communications Officers, Dispatchers, or Call Takers, their job is becoming increasingly more and more difficult as we pile on not just the volume of citizens that can reach out to them, but now the increased ways citizens can communicate, using nearly any device or modality.
Text to 911 ≠ NG911
Some industry leaders equate the rollout of Text to 911 to the evolution of NG911 multimedia communications. I’m not sure that I really agree on that. They often point out the painfully slow adoption rate by citizens, as well as the slow adoption of the technology in 911 centers nationwide. For me, this was no great surprise, as for nearly a decade we have stood by and watched as county by county PSAPs were added to the FCC’s master list. With over 6,100 PSAPs in the US, rollout was understandably slow, and the true benefit was never realized. Why? In my personal opinion, the deployment was controlled by the carriers and implemented as a business model, not a solution-based capability. The actual services were often limited, wrought with technical problems and lacked the attention to critical details such as location, one of the mainstays of the Public Safety industry. In fact, in most cases, location information was never actually delivered to the PSAP, and was only used by the network to decide the routing of the session to an appropriate PSAP.
“Hey Google . . . “
The latest innovation buzzword in the industry is AI, or Artificial Intelligence. But when you think about it, by definition, AI has actually been around for a long time, since the evolution of the computer. The very first mechanical computer or automatic computing engine concept was conceptualized as far back as 1822 by Charles Babbage. It was at that time Babbage began work on the Difference Engine, and it is considered to be the first automatic computing machine.
Today, many of us think of AI as being more than just an oversized calculator. Intelligence has a connotation of the ability to cognitively reason and then learn, providing a layer of self-governed decision-making capabilities where decisions are made based on fact and analytics and not emotional response. This brings to light something that is likely the biggest fear of AI naysayers . . .
The Terminator Paradigm
Admittedly, my biggest personal fear of AI is the Terminator
Paradigm. If AI was left to grow and learn unfettered by human intervention,
these super intelligent and logical entities would quickly come to the rational
conclusion that humans are dangerous to the growth and sustainability of the
planet, and therefore attempt to eliminate us. This is the entire premise of
the Terminator movie series, and it likely has some validity to it. Fortunately,
humans will likely remain in control and not implement full Artificial
Intelligence, but an iteration of it that I call:
Assistive
Intelligence – The New AI
The real value of Artificial Intelligence is the ability of technology to provide empirical decision-making guidance based not only on factual data points, but historical data. This is where ‘intelligence’ comes into play; taking past history into consideration and allowing the process of machine-learning to take place. Putting this in the premise of Public Safety and the call takers, no longer do they need to fear the possibility of being overloaded with information or inundated with so much data they will miss critical indicators needed for them to make a decision. The new AI, Assistive Intelligence, will help them sort through the masses of information presented to them, quickly locating the relevant bits of information that are buried within and germain to the present situation.
Re-spinning Technology for Tomorrow
One of the many valuable features that the Avaya Contact Center
offers is something called Whisper Page. This feature allows you to intrude on
another user and be heard by them without being able to hear the user’s
existing call which is not interrupted.
For example, as shown above: User A is on a call with user B. When user C intrudes on user A, they can be heard by user A but not by user B who can still hear user A.
The content of the Whisper Page can be audio from an AI Engine with specific advice to the call taker based on the context of the call as determined by a Speech to Text analytics engine.
One use case currently being developed by an Avaya DevConnect partner is the detection of a stroke condition based on specific speech patterns. A Whisper Page can alert the call taker to the condition, and then direct them to take on a specific list of questions suggesting the best course of action to take. The difference here is that the human (a.k.a. the Call Taker) remains in control of the situation and has the final decision-making authority. The AI tool is merely automating the Quality Assurance (QA) that is likely already taking place. What we have added is increased efficiency with the ability to do real-time QA providing instant feedback to the call taker.
The moral of the story? Understand and embrace technology. Allow it to contribute to your current capabilities and enhance them. Just think twice if the company name is Skynet.
This week, an article by Nicole Lindsey that was published in CPO Magazine, argued that citizen data privacy concerns outweigh the need for reliable location information being provided to first responders in the event of an emergency. This was in direct response to the Commission’s Further Notice of Proposed Rulemaking (FNPRM) for enhanced 911 location data.
In this FNPRM, US wireless carriers would need to start providing
vertical location accuracy when a user placed a cellular 911 call. The ‘accuracy’
that was proposed was to be within a height of plus or minus 3 meters to help
indicate the floor, or vertical ‘Z’ axis for users in multi-story buildings.
The goal to be met, was to require this enhanced location data in the Top 25
U.S. markets by the date of April 2021, and then a secondary milestone of the Top
50 U.S. markets 2-years later, by 2023.
The alarming position taken is one that incites mistrust in
our nation’s First Responders as well as the entire Public Safety infrastructure
as a whole. In her article, Ms. Lindsey states, “For years, the biggest telecom
network operators in the U.S. [. . .] have been secretly selling location data
of their customers”. While that claim is obviously true, it does sensationalize
the situation a bit, especially when linking the location data leaks to
emergency call traffic within the network. By doing this, the article unfairly
lumps our Nation’s 6,100+ Public Safety Answer Points (PSAPs) into a category
that is akin to telemarketers peddling back and knee braces, as well as low
cost health insurance plans and countless other scams being perpetrated on our
public in the rash of Robocalling activities currently afoot.
The last I checked, PSAP’s we’re not trolling the general public looking to push fire and ambulance services on unsuspecting citizens. Also, I’m fairly certain that anyone that was having an emergency that needed to call 911 would want to share their precise location information in an effort to save their life. Call me crazy, but I’m pretty sure most people would opt-in to that service. But, during an emergency, things happen quickly. Individuals don’t always or the ability to think with a clear head. It is for this reason that if a person indicates they’re having an emergency event, and they dial 911, it is likely a good idea to turn on location services if they’re off, and even a better idea to transmit that information to the PSAP so that the appropriate services can be dispatched to the correct and accurate address.
Yes, Robocalling, caller ID spoofing, and selling
location-based data for nefarious purposes should all be illegal and curtailed
using any means possible. But that should not impede or impact the ability for
legitimate public safety requests to be processed by the network and the PSAP
in a manner that provides them with detailed and discreet location information,
including altitude.
Let’s not throw the baby out with the bathwater. Let’s focus on the problem and deliver a workable solution that is not only efficient and secure, but one that protects citizen privacy in non-emergency events.
Simplicity. It is clearly one of the main tenets of any good Bill or Law. While 9-1-1 telecommunicators and dispatchers have been referred to as our Nation’s FIRST 1st Responders, their position has been relegated to one that carries the very same classification as an administrative or clerical worker.
Why is this important? Congressional Representative Norma J. Torres states, “Federal agencies rely on the Standard Occupational Classification (SOC), a vast catalog of occupations, for statistical purposes. The Office of Management and Budget (OMB) maintains the SOC. Occupations are supposed to be classified according to the nature of the work performed. However, the current version of the SOC categorizes Public Safety Telecommunicators as ‘Office and Administrative Support Occupations,’ which includes secretaries, office clerks, and taxicab dispatchers. OMB recently conducted a revision of the SOC but failed to appropriately update the classification of Public Safety Telecommunicators. “
In her eyes, and the eyes of many others, these individuals should be categorized as “Protective Service Occupations,” which includes a broad range of other “protective” occupations that are more closely related to the job at hand, such as: lifeguards, fish and game wardens, parking enforcement workers, firefighters, and even playground monitors, among others.
The 911 Saves Act is targeted at correcting this injustice and is one that YOUR Federal Legislator or Congressional Representative can get behind.
UPDATE FOR 3/6/19
March 7, Rep. Torres will formally introduce the 911 SAVES Act, which (if passed) will reclassify Public Safety Telecommunicators and dispatchers. This is a huge moment for everyone working in 9-1-1, and a great opportunity to have our voices heard. Here’s how you can help:
Get up to speed.First, read NENA’s original comments supporting reclassification for a quick refresher on the issue. Then, read up on Representative Torres’ bill in the 911 SAVES Primer.
Connect with your Congressional contacts. They’d love to hear from you on this issue. Email them directly and ask if their bosses would be interested in either cosponsoring or expressing support for 911 SAVES. Here’s a quick Grassroots Guide for reaching out!
Writer’s Block? Remember: your email doesn’t have to be too long or formal, but you can use this 911 SAVES Form Letter (here’s a version for your Senators) as a starting point for your correspondence.
Email and tweet directly at your elected officials. Use NENA’s new Online Action Center to reach out directly to your local congressional offices by clicking here.
Watch the livestream of the 911 SAVES press conference on Rep. Torres’ Facebook page. She’ll be introducing the bill Thursday at 3:30pm Eastern — click here to go to her Facebook page, where the livestream will be hosted.
In honor of his career, the Repack Airwaves Yielding Better
Access for Users of Modern Services Act of 2018 or the RAY BAUM’S Act of 2018
was raised in his namesake as a testament to his service to the American
people.
The RAY BAUM’s ACT has
picked up many small provisions where those issues on their own didn’t warrant,
or could not muster support for their cause in a separate bill. An item
particularly intriguing within this act, is in Title IV, under Section 506. It
states,
“The FCC must conclude a proceeding to consider adopting rules to ensure that dispatchable location is conveyed with 9-1-1 calls, including calls from multi-line telephone systems, regardless of the technological platform used. “Dispatchable location” means the street address of the calling party and additional information necessary to adequately identify the location of the calling party.”
While I am certainly NOT a lawyer, nor do I play one on TV, I can most certainly read and write the English language. The first 10 words of Section 506 say it all; “The FCC must conclude a proceeding to consider adopting rules”. That’s right, they have to finish making up their mind about having to make up their mind, or in other words, Get Ready to Get Ready. That’s it, end of the story – period.
Once again, the legacy database providers (a.k.a. the providers about to lose large revenues from database management fees) are running around telling their clients that the sky is falling, and they need to be compliant with a dispatchable location, pawning that off as individual station level reporting to the PSAP. Why? This seeds their coffers with revenue but actually provides very little actionable information to 1st responders.
For the record, I am NOT against providing detail to those
responding to an emergency. In fact, I am all for that practice. What I take
issue with is forcing a consumer to provide detail that is useless, at great
expense and hardship, only to create revenue for those who store the data for
public safety. Most often my arguments are deflected with the response, “Any
small level of detail can be helpful when trying to locate a person in an
emergency, and seconds count!”
Yes, seconds DO count. That is my precise argument. Instead of providing great detail that isn’t actionable (an EMT has no idea where cubicle 2C-231 is located in my building) why are we not using technology to create intelligent displays in the lobby that actually SHOWS a responder where they are needed, and how to get there in the event there is no one on site to guide them? The legacy Automatic Location Identification (ALI) record used today to convey information to first responders) is a text-only record just over 500 characters in length. There are minimal fields in there that provide the ability to include any relevant textual data.
NG911, on the other hand, is IP based and extensible. Information can contain text and URLs to additional data that can be retrieved dynamically if needed, and wherever it exists. The one problem that remains is the legacy voice network, capable of transmitting one thing and one thing only, VOICE. How do we get the data over this network?
SIMPLE: YOU CAN’T
We have been waiting for a decade or more for the NG911
ESInet to be built, but that is coming in dribs and drabs, and access is
anything but ubiquitous, so we did the next best thing. We took the information
we had in our SENTRY™ solution in the Shelby County Buildings Department and
delivered that information to RapidSOS during a 911 call. The call reached the Memphis
Police Department in Shelby County, and they were able to retrieve the associated
floor plans and information about the station that placed the 911 call.
ADDITIONAL, ADDITIONAL DATA
Along with floor plan information and emergency contact context, the SENTRY™ management console allowed text notes to be added to the incident record, and those additional notes were displayed to the call taker. History was made, and we moved the yardstick.
Of
course, I agreed and made my way down to Washington DC where I delivered my
presentation. I laid out my over-the-top delivery methodology for additional
data, where I effectively bypassed the voice carrier networks using the
Internet and releasing Ma Bell’s grasp and control of emergency services
location data, and it’s strongarm binding to pre-existing static location
records and phone numbers.
While
many saw the value of my architecture, of course, there were a few naysayers.
None the less, the idea itself was simple and quickly solved the problem of
getting data from the origination point to the resources that needed it; instead
of storing the information in a carrier-hosted database, where the subscriber
would not only have to pay for storage but maintenance as well as updates. By
placing a static, but unique, pointer in the carrier database, any queries
could be redirected back to the origination network. Not only would this remove
the excessive costs charged by the 911 location database providers, but the
actual information in the database would also now be available in real-time,
and be the most current available.
If any updates occurred, such as location change or a change of descriptive information, these would only be needed in the internal copy of the database. With this being owned and managed by the enterprise and entirely under their control, this model was much more efficient than the carrier-based model. The only piece missing was the connection to the PSAP, however by publishing the URL to the data in the Enterprise, Public Safety was able to reach out to the data when needed. The functional element on the enterprise side of this model filled the role of feeding the URL data and proved to be a practical and efficient solution. Based on this model, Avaya had the SENTRY™ emergency call management platform developed by 911 Secure, LLC as well as the associated integration modules. Now, enterprise networks could prepare for NG 911 services that were going to arrive shortly.
The
entire premise for this architecture was because the connection between the
originating network and the public safety answer point was an analog circuit
capable of voice communications only. What was missing and remained absent for
the next 6 years 2 months and 26 days, was a secure, high-speed connection
between the origination and the destination.
Earlier this year, RapidSOS announce the interoperability released with iOS 12 telephones. When those devices placed an emergency call, the location payload stored in the device would be transmitted to the NG 911 Additional Data Repository (ADR) being provided by RapidSOS. PSAP’s could access the repository by a standard query, once being vetted, and were able to retrieve the location of devices that originated emergency calls within their service area. Just a short time afterward, Google announced similar capabilities, also utilizing the RapidSOS repository. Within months of the availability, over 2000 PSAPs added the capability to their centers, covering nearly 70% of the population of the US.
Full
disclosure, for the past five years I have held a non-compensated position as a
technical advisory board member to RapidSOS. Because of this, I saw firsthand
the value this service brought to the table with this new national repository.
Since RapidSOS started to ingest data from any source through their published
APIs, I immediately went to work with the software engineers at 911 Secure,
LLC, the developers behind SENTRY™, and had them create an integration module
allowing the enterprise also to contribute location and additional data with
emergency calls. Within a few short weeks, they delivered a working model
placing data in the RapidSOS sandbox, and the search began for an Avaya
customer to be part of a live pilot program.
Fortunately, Shelby County Tennessee, a long time Avaya customer, was in the process of upgrading their CS1K communications platform to the latest Avaya Aura. Over on the public safety side, Shelby County 911 had just implemented the embedded RapidSOS capability in there Motorola VESTA™ platform, as well as the RapidSOS Lite web-based functionality in the PSAP serving the County facility. After presenting our use case to both parties, we began installation of SENTRY™ just before the holidays.
Finally, the day of reckoning came. On January 18, 2019, 2278 days after I presented my over-the-top architecture to The Federal Communications Commission, a live call to 911 was placed from the Shelby County Buildings Department, answered at the Shelby County 911 Center, where they received Voice, precise location, and additional data in the form of floor plans. There it was, we made public safety technology history. I couldn’t help but feel a sense of pride, as we changed the game forever, and proved that NG911 was not only possible but a reality.
This past
year I had the honor of being in the Oval Office with Hank Hunt as the
President signed Kari’s Law into the law of the land, I was invited to be part
of the Haleyville Alabama 50th year 911 Day celebration and be a Grand Marshal
with several of my good friends in the Town parade. Now, I was a part of
telecommunications history as Avaya, 911 Secure, RapidSOS, Shelby County
Buildings, and Shelby County 911 worked in concert to enable the very first
emergency call delivering NG 911 additional data to the PSAP.
Not only will this technology help save lives but provide desperately needed location details to public safety first responders, as well as critical multimedia such as video and still pictures in the event of an emergency.
A few of you may remember, back in July 1969, what was then
be most famous, and furthest, Long distance phone call ever made. As for the
rest of you, you are now Googling of phones even existed that long ago!
I can assure you that they did, and on July 20, 1969, then President
Nixon spoke with crew members Neil Armstrong and Edwin “Buzz” Aldrin
via telephone-radio transmission, with the President in the Oval Office and the Apollo XI
astronauts Neil Armstrong and Edwin “Buzz” Aldrin while they were on
the surface of the Moon.
Of course, that call originated on landline circuits, that is upconverted to a satellite link and then beamed into outer space on the Goldstone Deep Space Network. In many ways, this radio transmission is capable of voice and data, similar to any terrestrial based radio transmission. We’ve modern advances in communications, just like we have Wi-Fi here on the surface, the International Space Station (ISS) is also connected.
The magic of VoIP allows any IP-based telephone to exist no
matter where the connectivity is coming from. That being said, it was really no
amazing feat to put an IP phone inside the IIS, which apparently was done a few
years ago. Unfortunately, IP phones don’t live on their own, they need to
register and connect with a call server that provides trunk resources to the
outside world. Once again, our space based VoIP phone follows this same rule,
and is connected to an IP telephony system inside NASA headquarters.
As many people do, when calling international numbers people
forget to dial the zero in the 011 International prefix. On the ISS phone, one
of the astronauts recently dialed ‘9’ for an outside line, forgot the ‘0’, and
then dialed ‘1 1’ followed by an international number. Of course, being a KARI’S
LAW compliant telephone system, as soon as the system processed 911, the call
was sent to public safety triggering internal alarms along the way.
Fortunately, everyone realized it was just an accident, and
there was no emergency launch of a police cruiser to intercept the IIS in orbit!
So what’s the lesson learned? 911 needs to work everywhere, including “up there”!
But, it might be a good time to put in a Little missile prevention programming J
For as long as I can remember, we built an engineered networks for “3 R’s”. Resilience, Redundancy, and Reliability. Following the simple rule would protect you from the 50-year flood, the 100-year storm, and many other “rainy day outages”. When the network was up and running, humming away and performing nominally, it was considered a sunny day. Systems were online, everything was running well within specification, and network administrators would sit and babysit their huge collection of silicon and copper wires.
Every once in a while, the skies would become cloudy,
network elements would fail, conductivity would be lost, and the data center
would run at something less than full capacity. While this was certainly
something that needed to be addressed, stress levels remained tolerable as that
engineered Resiliency, Redundancy, and Reliability were all there allowing data
to be processed with little to any notice outside of those directly responsible
for the systems uptime.
Before the evolution of the Internet, and the acceptance of cloud-based services in massive data centers, most facilities manage their own data centers where they had full control over the building, environmentalists, and even diverse carrier network connectivity. Despite this, the IT “Big Bang” (a.k.a. the Internet) occurred in considerably shook up the model. Massive data centers sprung up around the country out in the middle of farmlands that housed thousands and thousands of servers, virtual machines, and facilities for nearly every industry.
There was no mistaking it, the cloud was here and everyone was in it. Performance and capacities rivaled that of localized data centers, and with the proper design, a mesh environment could be established where even if a portion of the network did go off-line, several other nodes were standing by the ready to pick up the slack. Too many, we finally reached a utopia of computing power, and more and more critical applications were perfectly comfortable sitting in the public or quasi-public cloud.
Most of the time, I try to keep my thinking simplistic. I
like to go back to the basics and understand the fundamentals of just about
anything that I do. I believe if you truly understand, at a very deep level,
how a certain process operates then when that process fails your equipped with
the capabilities to properly troubleshoot, repair it, and, most importantly,
design around a similar failure in the future.
The most recent victim over the New Year’s holiday was the CenturyLink network. News reports over the weekend noted that areas of the country including Idaho New Mexico and Minnesota were affected as well as residential services in 35 total states. 911 services were also affected across the country which prompted nationwide alerts to cell phone users advising them to utilize local 10 digit numbers in case of an emergency. Initial signs of the outage were detected around 1 AM Pacific time Thursday morning with the resolution being achieved by approximately 6 PM Pacific time on Friday for a total of about 41 hours.
Five nines reliability??
When we build networks, we strive for five nines reliability, or 99.999% uptime calculated on an annual basis. Mathematically this works out to be just over five minutes of outage allowed per year. Based on 41 hours of disruption in the CenturyLink network, they’re starting off a brand-new year already down to just 2 1/2 nines, or 99.531 by my calculation.
So let’s look a little deeper into this particular failure,
keeping in mind that the full root cause analysis is still likely a week or so
away.
RESILIENCE
In many cases, resilience is not a specific thing. Resilience is the ability to step back into action when any particular outage occurs. It doesn’t define what that action should be, only that it was quickly identified and remediated. So while it’s a bit nebulous, you might say that resiliency is likely one of the most important pieces of any recovery plan. Contingencies are expected, spare parts are readily available, and monitoring tools have been deployed to quickly isolate problems, as well as the training and skill sets of personnel to utilize those tools and carry out any remediation tasks.
RELIABILITY
When we talk about reliability, we talk about the confidence level that a particular component will perform nominally. For example, an incandescent light bulb may operate for up to 2000 hours, but new LED replacement lamps are routinely quoted as 50,000 hours of operation, making those lamps 25 times “more reliable”. In telecommunications networks, if you cannot increase in individual components reliability, utilizing a high-availability, the active-active model can ultimately achieve the same goal. If one processor fails, the other processor is already running taking over the operation. This shouldn’t be confused with active – standby, where there is still a disruption, although minimal, as the secondary processor comes online. The critical component here is the detection of the failure and the redirection to the secondary processor.
REDUNDANCY
There is always strength in numbers. Redundancy goes hand-in-hand with both resiliency and reliability. Nothing ever lasts forever, especially electronic components. We try to calculate an MTBF (mean time before failure), however, those numbers are usually unrealistic for day-to-day operations as there are many contributing factors at the subcomponent level that could cause catastrophic failure.
The magic to running a solid and stable network is to closely manage, monitor, and statistically analyze every possible metric that there is. When a failure or does occur, careful root cause analysis must be undertaken to determine what in fact failed, but it doesn’t end there. Taking it a step further and understanding the key indicators that were present prior to that failure are going to be what help you proactively divert that failure in the future. Burn me once, shame on you. Burn me twice, shame on me. Sunny day outages are the new uptime threat.
