Click here for the Audio Podcast of this Blog

For as long as I can remember, we built an engineered networks for  “3 R’s”. Resilience, Redundancy, and Reliability. Following the simple rule would protect you from the 50-year flood, the 100-year storm, and many other “rainy day outages”. When the network was up and running, humming away and performing nominally, it was considered a sunny day. Systems were online, everything was running well within specification, and network administrators would sit and babysit their huge collection of silicon and copper wires.

Every once in a while, the skies would become cloudy, network elements would fail, conductivity would be lost, and the data center would run at something less than full capacity. While this was certainly something that needed to be addressed, stress levels remained tolerable as that engineered Resiliency, Redundancy, and Reliability were all there allowing data to be processed with little to any notice outside of those directly responsible for the systems uptime.

Before the evolution of the Internet, and the acceptance of cloud-based services in massive data centers, most facilities manage their own data centers where they had full control over the building, environmentalists, and even diverse carrier network connectivity. Despite this, the IT “Big Bang” (a.k.a. the Internet) occurred in considerably shook up the model. Massive data centers sprung up around the country out in the middle of farmlands that housed thousands and thousands of servers, virtual machines, and facilities for nearly every industry.

There was no mistaking it, the cloud was here and everyone was in it. Performance and capacities rivaled that of localized data centers, and with the proper design, a mesh environment could be established where even if a portion of the network did go off-line, several other nodes were standing by the ready to pick up the slack. Too many, we finally reached a utopia of computing power, and more and more critical applications were perfectly comfortable sitting in the public or quasi-public cloud.

Most of the time, I try to keep my thinking simplistic. I like to go back to the basics and understand the fundamentals of just about anything that I do. I believe if you truly understand, at a very deep level, how a certain process operates then when that process fails your equipped with the capabilities to properly troubleshoot, repair it, and, most importantly, design around a similar failure in the future.

The most recent victim over the New Year’s holiday was the CenturyLink network. News reports over the weekend noted that areas of the country including Idaho New Mexico and Minnesota were affected as well as residential services in 35 total states. 911 services were also affected across the country which prompted nationwide alerts to cell phone users advising them to utilize local 10 digit numbers in case of an emergency. Initial signs of the outage were detected around 1 AM Pacific time Thursday morning with the resolution being achieved by approximately 6 PM Pacific time on Friday for a total of about 41 hours.

Five nines reliability??

When we build networks, we strive for five nines reliability, or 99.999% uptime calculated on an annual basis. Mathematically this works out to be just over five minutes of outage allowed per year. Based on 41 hours of disruption in the CenturyLink network, they’re starting off a brand-new year already down to just 2 1/2 nines, or 99.531 by my calculation.

So let’s look a little deeper into this particular failure, keeping in mind that the full root cause analysis is still likely a week or so away.

RESILIENCE

In many cases, resilience is not a specific thing. Resilience is the ability to step back into action when any particular outage occurs. It doesn’t define what that action should be, only that it was quickly identified and remediated. So while it’s a bit nebulous, you might say that resiliency is likely one of the most important pieces of any recovery plan. Contingencies are expected, spare parts are readily available, and monitoring tools have been deployed to quickly isolate problems, as well as the training and skill sets of personnel to utilize those tools and carry out any remediation tasks.

RELIABILITY

When we talk about reliability, we talk about the confidence level that a particular component will perform nominally. For example, an incandescent light bulb may operate for up to 2000 hours, but new LED replacement lamps are routinely quoted as 50,000 hours of operation, making those lamps 25 times “more reliable”. In telecommunications networks, if you cannot increase in individual components reliability, utilizing a high-availability, the active-active model can ultimately achieve the same goal. If one processor fails, the other processor is already running taking over the operation. This shouldn’t be confused with active – standby, where there is still a disruption, although minimal, as the secondary processor comes online. The critical component here is the detection of the failure and the redirection to the secondary processor.

REDUNDANCY

 There is always strength in numbers. Redundancy goes hand-in-hand with both resiliency and reliability. Nothing ever lasts forever, especially electronic components. We try to calculate an MTBF (mean time before failure), however, those numbers are usually unrealistic for day-to-day operations as there are many contributing factors at the subcomponent level that could cause catastrophic failure.

The magic to running a solid and stable network is to closely manage, monitor, and statistically analyze every possible metric that there is. When a failure or does occur, careful root cause analysis must be undertaken to determine what in fact failed, but it doesn’t end there. Taking it a step further and understanding the key indicators that were present prior to that failure are going to be what help you proactively divert that failure in the future. Burn me once, shame on you. Burn me twice, shame on me. Sunny day outages are the new uptime threat.

It’s Sunday night in New England, and once again and I find myself checking into a hotel getting ready for customer meetings several hours away from home.

Tonight, I’m checking into a Homewood Suites by Hilton, in Warwick Rhode Island. Like most Hilton properties, this franchise is very clean, fairly new or recently renovated, and digitally transformed with their App based check-in process, and digital room keys. As I pull into the property, my Hilton App pings me and reminds me that my digital room keys are now active on my phone, and I only need to stop by the front desk to pick up my complimentary bottles of water, and Milano cookies (my favorite part!)

As I park my car, I remember thinking; newer property, well-maintained, new digital locks. I am going to give this property an 80% chance of being Kari’s Law compliant.

Upon entering my room, I am greeted by the standard TeleMatrix Hotel room phone, and a faceplate that says, ”EMERGENCY Dial 9–911”. While many of my Kari’s Law fact checkers out there will immediately point this out as a noncompliant hotel, I shrug this fact off, as the face plate on the phone means nothing.

The ability to actually dial 911 directly is what really counts. But, what it does mean, is that now there is some investigative work that needs to take place to find out what the real situation is.

As I go to leave my room to go downstairs and speak with the night manager, I noticed the sign on my door that says:
”CALL THE FIRE DEPARTMENT  911”

Uh-Oh! NOW we have a problem! There is a sign that specifically says to dial 911 and a phone that is labeled 9-911. This means that both ways better work.

As I reach the front desk, the night manager (we’ll call her Allison) greets me and asked if she could be of any assistance? I sigh slightly, warning her that this is going to be a complex and long-winded story, but I needed to ask her a few questions about 911. Immediately, she made sure that I didn’t need to call 911, which I thought was a great response to my statement, and clearly, this woman had a game plan formulating in her head already. I reassured her there was no emergency, and I explained Kari’s Law to her. Allison then told me she never heard of Kari’s Law, but knew about the requirement to directly dial 911, on-site notification, and began to show me the process.

While many Hilton Hotels use Avaya or older Nortel Systems, there are some that do not. This particular franchise had a competitor system installed, that I would guess to be six years old or more. Despite the age of the system, Allison showed me how the console would indicate the extension number of any phone dialing 911 would display to her, letting her assess the situation.

She then pointed to another phone on the counter, that was marked, “EMERGENCY PHONE FOR 911 CALL BACK – DO NOT USE”

Allison then explained that the Warwick Police Department would call that phone whenever they received a 911 call from the hotel. This call was placed while a Police unit was en route to the location, and the 911 call taker would attempt to ascertain any additional information as well as the status of the emergency updating the responding resources accordingly.

If it was a false alarm, the police would still respond to file a report and be certain there was no need for public safety.

I thought to myself, “what a great example of exactly what Kari’s Law was designed to do! Focus on DISPATCHABLE LOCATION or address the units needed to respond too”.  The room is not relevant, that information can be gathered on-site, verbally or electronically through a simple display mechanism, as well as additional information that is germane to the incident.

In this case, awareness was raised, a plan was established that met the needs of local public safety, as well as the hotel property, and despite the cries of 911 database providers who are losing their billable revenues, these simple plans work. Nothing in this plan burdens MLTS operator with costly upgrades, monthly billing of 911 services, and complexity. The solution is just a simple workflow response to a life and death problem, and one that would have likely saved Kari Hunt’s life on that East Texas December 1st in 2013, as per the Coroner’s testimony.

Hotels, businesses, schools, hospitals or any other facility with an MLTS now know this problem exists. And the fix is simple in most cases.

This story comes at a perfect time. The Federal Communications Commission has an active Notice of Proposed Rule Making under FCC docket 18-261. At this very moment, the various stakeholders are frantically jockeying for position hoping to get their revenue-based model codified as law when it is simply not needed. As Hank Hunt so eloquently states in his September 26, 2018 testimony, these folks are, “nursing a bottom line like a little baby.”

I so wish that I could speak like that, but Hank Hunt has me pinned to the mat with his command of the English language and ability to convey his deepest thoughts and feelings. Kudos Hank! You got me beat like Kansas City barbecue being served in Winona, Texas! (not too bad for damn Yankee, huh?)

While we’ve made great progress over the past five years, now is not the time to sit back on our laurels. The important legislation is being discussed that directly affect how this law is implemented. There are those that are looking to fix the problem, and many others looking just to profit from it. I suggest you take the time and read the various responses to the Federal Communications Commission’s NPRM, as well as follow the reply comments that will be coming in over the next 30 days.

Anyone can go to http://www.fcc.gov/ecfs and search under Kari’s Law or 18-261. And remember, ANYONE can comment. Additionally, ANYONE can file a reply comment even if no initial comment was filed. In fact, you can listen to a recent podcast with FCC CTO Eric Burger, recorded at the recent Real-Time Communications Conference held at IIT, the Illinois Institute of Technology.

Follow me on Twitter @Fletch911
Listen to my PODCASTS on – Avaya.com/APN

One of the universal truths for all 9-1-1 centers is that there will be new people.

In the fire service and some dispatch agencies, these are referred to as “probies”, short for Probationary Employee. Whatever they are called, however, they are the most important resource your agency has, unless you want to continue answering phones until you are well into your eighties or beyond. After all, we can’t keep doing the job forever, so we need to make sure there is a next generation to replace us.

Sadly this process doesn’t go as easily or smoothly as we would like it. An entire book could be written on the process of recruitment, hiring, training, evaluation, and retention (and perhaps it will be). But I would like to keep this short so let’s just focus for a minute on some critical things that the new person needs to know and the people already on the job need to know. If you have additions to this please share.

To Those Already On The Job:

The new person does not arrive fully formed with everything they need to know bestowed by the training process, their life’s experience, or time spent watching Chicago Fire, COPS or even CHiPs. They will only be a successful employee if you provide an environment for them to be successful. Even if you have never seen yourself as a trainer, CTO, or anyone close to having official mentoring responsibilities, the truth is they need you to help them. Be patient, be open-minded, but be firm and fair in the process. Understand that how you treat them in their first weeks and months on the job will have more impact on their career than you can ever imagine.

Do not use the way you were treated as the road-map for how you should treat your new person. Use instead the road map of how you SHOULD have been treated. In dealing with adults this means treating people with respect, appreciation, and understanding. It is not easy to learn new things. Does this mean that every new person should always stay in the job, of course not? Some people will wash out, that is just the way it is. Sadly, many good dispatchers are lost to the poison of bad coworkers. Either they don’t accept them, don’t help them or just don’t care. We are better than that.

If we are a profession, then it means we should support those who come into the profession behind us. Be a resource, be helpful, be kind, and show by example what a great dispatcher can and should be. Even if you never work directly with the new hire, leading by example is a powerful tool for creating our next great generation of Dispatchers. Do your part. Please.

To The New People:

Welcome to one of the world’s greatest professions. You are now part of a family, which is a good thing and a bad thing. Your feelings will get hurt. Get over it. Your opinion may not be welcomed, get over it. You will be expected to be the first person in the center every shift and the last one to leave. Get over it. You will have to learn to quietly listen and take in all that goes on around you, interjecting only when critically necessary. That is as it should be. Accept it. You will be expected to learn and study when others may be taking a break. That is the way it should be. You will never gossip about your coworkers, tell stories “out of school” or badmouth people you work with. Make this truth the first and last thing you say to yourself each day. If you can do these things, the people you work with will come to trust you and you will become a part of the team. The people you work with will become like family. They will help you, support you, take care of you, look out for you and your life will be far more “rich” for knowing them.

This will not be because you completed all the tasks on the checklist. It will not be because your words and opinions prove your worth to the organization. No, your actions and your approach to your job will show others (and yourself) that you belong. Imagine a baseball player who got the highest salary but never hit a home-run. Or a chef who never stepped foot in a kitchen? It’s not what you say that earns you trust or builds your reputation, it is how you work. Show your fellow dispatchers you care by your actions.

Take the time to learn and never stop. Go to the weddings and the funerals. Sign the birthday cards. Bring in food for the shift once in a while. Take part in the important rituals that are part of any great organization. Most of all, learn how to be a great dispatcher. We learn not by speaking, but by listening, watching, showing an interest, and asking questions. This is hard. In this day and age, we want to share our opinion, to stake our claim. This will not endear you to those you work with. In public safety we must often act based on intuition, just knowing what is going on and what is required. In order for intuition to be effective, trust is required. Do everything you can to build that trust– trust of yourself and trust of others– and you will soon find out what many of us have known so long, being a part of the 9-1-1 world is one of the greatest jobs you will ever know.

C. Carver, ENP

---

This classic, and timeless post by Christopher Carver, ENP, is one that I am proud to put on my little space in the interwebs, for your education and enjoyment.  Chris has had a long and reputable career in Public Safety, whose actions and guidance to others likely saved countless lives that will never know or appreciate. Early in 2015, Christopher Blake Carver joined NENA the new PSAP Operations Director. In this role, Carver oversees projects related to current, emerging, and prospective issues affecting 9-1-1 centers. He is someone I consider not only a close colleague in the industry, but a true friend.

 Follow me on Twitter: @Fletch911
  Read my Avaya Blogs: Avaya.com/Fletcher
  Listen to my Podcasts: Avaya.Com/APN

 

It’s no great secret that the existing E911 infrastructure in the US is well overdue for an overhaul. Does it need an upgrade? NO.. Does it need a refresh? NO..  Does it need a complete overhaul? Unequivocally, YES.

On February 16, 1968, technicians from the Haleyville Phone Company in Haleyville Alabama decided to take on and challenge Ma Bell to a race. They worked tirelessly to design and deploy the very first 911 system in the country. It wasn’t necessarily a technological revolution,  by today’s standards, but it was incredibly disruptive thinking applied to a problem that needed to be solved. Over the years, updates and upgrades were placed on our telephone network. These added capabilities like Selective Routing in the Northern Telecom DMS-100 Digital Switch Platform, as well as using CAMA trunk technology (borrowed from the Hotel/Motel industry to report room long distance to lodging operators) to provide the originating number to the PSAP operator, as this was long before the invention of Caller ID.

In the network, call routing to the right 911 PSAP was based on telephone numbers, and Ma Bell being the keeper of telephone numbers and their installation addresses. At the time, this became the de facto standard for ‘location’ discovery and proved to be very accurate. This worked well for years, and again, the technology was patched and band-aided to add capabilities like PSAP to PSAP call transfer, at least within localized areas of the country.

Then, in the mid 80’s a disruptive new communications technology reared its ugly head. That technology is what we all now know and understand as the cellular telephone. This fantastic technology provided mobility to those that had the service, initially vehicle bound, but soon these devices gained legs and were portable. First introduced as ‘bag phones,’ handheld devices became small and cheaper, and within 15 years, this new trend in communications became well entrenched in our lives.

Fast forward to today, the cellular penetration has exceeded 100% in the US, in fact, the latest report from the CTIA put the figure at 117%. While this fantastic new technology developed and flourished, the 911 emergency network remained stagnant with little to no improvements. Even the devices no longer had a fixed location, the Legacy model of the telephone number to street address correlation remained in place despite its growing uselessness.

When we look at modifying the existing 911 network to utilize the vast data that smart devices bring to the table today, everyone searches for the magical box that will convert old technology to new. Want to talk about this publicly, I always warn the crowds that this is the part where I need to use the ‘F-word,’ so please be ready and don’t be offended. There is the magical box. There’s no magic wand that I can wave to transform old to new. If you want to change the existing network to a new next-generation network, (get ready for the ‘F-word’), you’re only going to have to ”Forklift” what you have today. I know that term is not well-liked by many in the technology industry, but it is, unfortunately, a reality.

At least some of my readers will still remember growing up with the black-and-white television. At some point in time, Color TV programming became available. But until we replaced that black-and-white TV with a color set, it would be impossible did you color content. If a television station New York City utilized a black-and-white television transmitter, even if they were transmitting programs that were recorded in color, anyone receiving their signal would be restricted to a black-and-white transmission. Until they took that black-and-white transmitter and replaced it with a color transmitter, their signal would be stuck in the black-and-white world.

Now, one may argue that the black-and-white television or the black-and-white TV transmitter could be disassembled, and then rebuilt with new color components. While that may or may not be true, it is most certainly the long way around, and likely fraught with problems. This is precisely where the Band-Aid approach has brought us today.

I often wonder why the carriers and 911 database providers take this approach. Well you understand there’s a financial advantage to tying a telephone number to every device, and then creating a database for that device, it’s incredibly inefficient and limits functionality and the ability to move forward. With no technological advantage, the only alternative left is revenue, something that carriers desperately hang on to, like it’s their lifeblood.

I was recently a member of a panel discussing real-time text (RTT) at the Federal Communications Commission. During that, the question came up about location capabilities with RTT. The way text to 911 is delivered today, location is often omitted and not available to the 911 call taker. It seemed to be a legitimate question since RTT is a form of digital IP communications, the originating device, which is well aware of its location, now has the ability to communicate that information directly with the PSAP. But, I was shocked to hear the representative from a large 911 database company respond with, “we’ll use the same mechanisms that we use today, and communicate location through telephone numbers.” After hearing that response, I nearly dropped to the floor. Here we have an opportunity to correct one of the biggest problems affecting Cellular devices and 911 calls. And we have a technology that is enabling a peer-to-peer IP connection that will allow passing the explicit data from the device to the dispatcher, yet we’re going to opt for antiquated technology that is ineffective for the mission. This would be like filming a movie in high definition Color, and then buying a high definition color TV to watch it on, but sticking a black-and-white TV transmitter in the middle to broadcast the signal.

What I listened to FCC commissioner Jessica Rosenworcel comment a week earlier on implementing Kari’s Law, there is one part of her statement that ring true. She reminded everyone that not only do we need to correct this particular problem, but we need to set a deadline of when the work would be completed. I believe by establishing specific set timelines, in applying penalties for those that failed to meet them, we’ll change the financial model to be one that will support the upgraded and refresher technology, and we won’t be left with such a dichotomy of Origination device capabilities, and the network’s ability to carry the new data.

This is not rocket science. Many of the basic principles applied by commercial enterprises apply to public safety. In fact, the only difference is the amount of resiliency and redundancy apply to the architecture. The workflows are similar, if not the same. We’ve solved the multimedia problem for the airlines, Financial institutions, and commercial retail businesses. For them, it was a competitive advantage. Their customers were made up of the same Group of citizens that want to contact emergency services through new digital channels. Large commercial enterprises were faced with a forklift of their current technology or risk losing their customer base. Now, Public Safety isn’t that same decision point. Except, they won’t lose customers, they’ll lose lives. And that is something none of us can live with.

Follow me on Twitter @Fletch911
Check out my AVAYA CONNECTED Blogs

On Monday, September 17, 2018, the very first nail in the coffin of the legacy 911 network will be firmly planted and driven home. Apple is set to release iOS 12 that contained several new features and enhancements, one of them being “EES,” Enhanced Emergency Services. In a white paper published in August of this year, Apple announced their Hybridized Emergency Location (HELo) technology providing precise, high integrity location data to 911 centers.

The white paper states, “Apple devices contain a variety of location sensors. […] Apple devices can “fuse” information from various sensors, such is [GPS] and Wi-Fi. [The technology provides] a low uncertainty, high integrity estimate of the devices location.”

As it was in the past
HELo is a radical and revolutionary step forward. Currently, the only source for location on a cellular device has been the wireless carriers, who provide this location information through often very inaccurate triangulation calculations. These give much less precise location information, or even worse, merely the cell tower handling the call. This causes a significant problem for those who cannot communicate precisely where they are because they either don’t know or are not capable of speaking. This is a problem for the deaf and hard of hearing community, as well as for individuals with speech disabilities. These individuals are forced through relay centers, outside of the 911 network, that have zero visibility into any location information.

As it will be going forward
HELo is the United States adaptation, of a technology that’s been operating in the European Union for a few years, called AML (Advanced Mobile Location) and operating on Android devices since last year. The concept is simple and initiated initially by John Medland at BT in England. HELo, AML operate nearly the same, conceptually, and even the European eCall (the EU version of OnStar) initiative follows a similar approach on the backend consumption of the data.

How it works

When your cellular device initiates an emergency call, the voice call is sent to the legacy 911 network as it is today. The wireless carrier uses the cell tower information to route your request to the closest Public Safety Answer Point or 911 PSAP. At the exact same time, HELo takes the location data from your phone (the precise same location data that Uber and Domino’s uses in their app) and places that in a National NexGen 911 Clearinghouse Data Repository with your telephone number as the index reference. When the 911 center gets your emergency call, they initiate the standard database queries to the carrier and are displayed the legacy location parameters, but many 911 desktop application providers have added the functionality to immediately make a secondary query to the National NexGen 911 Clearinghouse Database. The location from that response is often exact and considered to be “high fidelity”. The 911 call taker gets a second plot displayed on their display along with the original location accuracy estimates from the carrier.

Where it will work
While native support for EED will be available to compatible Apple devices when they do the free software update to iOS 12, some carrier networks may block or impair the service from operating as designed. For example, not all networks support simultaneous voice and data. So, if a carrier prevents the outbound data connection, the device would only be able to communicate the location payload over a Wi-Fi connection. I experienced this phenomenon two years ago when Avaya was testing similar technology being accessed through HTML 5. Our initial tests went perfectly, but we happened to make a check in an area where my connection was downgraded from LTE to 3G. Under these conditions, my carrier blocked external data connections during a voice call that was an emergency. It was a perfect example of legacy thinking bubbling through current technology impeding innovation. I had to actually file formal complaints with the FCC against several carrier networks to get this practice changed and removed, under the basis that it had no merit in today’s environment.

Privacy – Everyone’s concern
I am sure that Apple is going to maintain confidentiality and security as a core value. Apple has stated that they will take “extra steps to ensure that our products and services protect the confidentiality, integrity, and availability of our users data during an emergency call.” to enforce this stance on the issue, Apple plans to use Geofiltering to minimize the potential for disclosure even to trusted parties that are not associated with the incident. Also, if the PSAP servicing the user’s location has not opted in to receive this new information, the data is dropped and not stored.

Security – Data Encryption
Data between the user’s device, the RapidSOS clearinghouse database, and the PSAP is all encrypted with strong ciphers and long keys. Additionally, the data remains encrypted while in transit, and even at rest in the databases.

Data Longevity
To prevent a future data breach from yielding information about previous events, Apple discards all data that fails to match geolocation criteria of a PSAP servicing that location, and all data is deleted after 12 hours. Any information that is sent to the RapidSOS clearinghouse database follows the same guidelines. Any data that is ultimately transmitted to a PSAP, state and local records retention laws apply and are up to the receiving agency to enforce as with all data they received today.

Thanks – Not for me . . .
I sat racking my brain trying to figure out a reason why you would NOT want this service activated, but unfortunately, came up blank. Even with that being the case, Apple has provided an opt-out capability, and EED, although enabled by default, can be easily deactivated and disabled in the settings app of an iOS device at any time.

I’m excited
While this new capability is currently limited to Apple iOS 12 devices, this puts in place a key component for next-generation emergency services in the United States. The National NG 911 Clearinghouse, officially known as an ADR (additional data repository) is a crucial element in the transition from the legacy network to next-generation 911 capabilities for all devices. This element must exist for originating devices to place their data in and will serve as a DMZ boundary for PSAP’s to reliably trust adding explicit information to emergency call events.

While the legacy 911 databases remain an essential source of information today, the implementation of architectures like this allows intelligent networks and devices with relevant information not only about the location but situational awareness of the environment, to now have a mechanism and pathway to public safety first responders that can utilize this information.

Knowing that Mark Fletcher sits at cube 2C – 231 is irrelevant information that isn’t actionable because first responders have no idea where that is without a map of my building. Now I can provide them with a floor plan, as well as the fact that it’s 185° at that location. Hmmmm . . . That’s a little warm. Maybe something’s on fire?

That’s actionable data, and worth formulating a system to collect, correlate and propagate to public safety.

Follow me on Twitter @Fletch911
Check out my AVAYA CONNECTED Blogs

 

To listen to an audio podcast of this blog  [  CLICK HERE  ]

More than four years ago, I published an article on my Avaya CONNECTED blog written by my friend and colleague, Ty Wooten, ENP. Ty is NENA’s director of PSAP Operations and Training, and he along with Maureen Will, director of Emergency Communications at Newtown, Conn., contributed a great article on school safety.  Since that time, the relevance and importance of communications between 9-1-1, local first responders and school officials have increased even more.  Every month brings a new tragic shooting, often in a school. The lessons learned on how to minimize the impact of these events remains the same – improved communication. Recent articles out of Nassau and Suffolk Counties in New York are focusing on the deployment of a software panic button technology there, and this has an intriguing overlap with E9-1-1, Kari’s Law (first enacted in Suffolk County), and the evolution of Next Generation 9-1-1 (NG9-1-1) technologies.

The Communication Gap

It’s well-documented in several studies that most active shooter incidents are over within 5 minutes to 12 minutes.  Likewise, the average response time for public safety personnel ranges from 13 minutes to 18 minutes. The Naval Post Graduate School, the State of Massachusetts, the FBI and NYPD research projects all point to two key factors in reducing the casualty impact in these incidents:

1. Improved communications
2. Victim-initiated notifications

What “victim initiated” basically boils down to is the person who is closest to the situation initiates the alarm as quickly as possible, notifying authorities as well as any collateral population, such as faculty and students.  Confusion and procedural missteps can occur when a call for help gets intercepted and is“triaged” by untrained staff. One has to ask “Why?!”  It seems that all too often systems notify the administrator or front desk, where it is expected that the call will be answered and evaluated, and then responders are notified.  This scenario, to the uninitiated, may sound efficient but it’s a stark reminder of how hotels configured their PBXs to intercept calls to 9-1-1 internally. As a result of Kari Hunt, a mother of three died in 2013. Why? A failure in communications. 9-1-1 was not able to be dialed, and no notification was made to anyone.

The Rave Panic Button, which is a software solution being deployed in Long Island, N.Y.,  took the basic concept of an on-site notification that is inherent in Kari’s Law and applied it to cellular phones (where 80 percent of 9-1-1 calls originate from). 9-1-1 remains a critical communication point and needs to stay involved to properly coordinate the response. Simultaneously, individuals on-site get situational awareness around the event and can initiate appropriate preparations.  In line with this, regardless if 9-1-1 is dialed from the PBX, natively on a cell phone, or a physical device is activated either from a dedicated panic button on the wall, or an app that simulates the same capability directly, on-site notifications are sent to the designated individuals with situational awareness, while the call is directed to the 9-1-1 PSAP.

As the call is answered by 9-1-1, notifications are sent to the Rave app and update the status.– This step alone is valuable in reducing mass call events and call overload at the PSAP.  Finally, and perhaps most importantly, the call taker now has an intelligent link and the ability to push follow-on information and notifications to personnel on site. As Maureen and Ty highlighted in their article, 9-1-1 is essentially incident command for the duration of these incidents. The ability to manage all resources, including providing critical updates to individuals on-site can be invaluable.  Just imagine, if 9-1-1 had been able to provide notice to those evacuating the Parkland school that the fire alarm was false and just a diversion and that they should remain in lockdown, lives may have been saved.

 

Does this fit with NG9-1-1?

Without a doubt, I am one of the biggest proponents of Next Generation 9-1-1, but it amazes me that I still find people that feel (coupled with FirstNet), NG9-1-1 will automatically solve all of Public Safety’s communication challenges, and the world will now have rainbows and unicorns. SORRY,  that’s NOT going to happen.  While NG9-1-1 will be an enabling technology providing a base framework for powerful applications, it is not in itself the answer to the problem. Take, for example, the additional location data repository and additional caller data repository that are functional NG9-1-1 elements. The challenge is that while NG9-1-1 specifies these elements, outside of the Avaya SENTRY™ solution, we rarely see anyone providing a source of additional data for public safety’s use.  While some municipalities have gone through the exercise of collecting floor plans, these plans are usually gathered at the time of construction and are rarely updated. Data is good, but inaccurate or old data is useless. That being the case, while NG9-1-1 and FirstNet provide the architecture and pipeline to get data to first responders, we still need a source of GOOD DATA to make any impact on the operational effectiveness and response.

 

There are examples of CAD systems supporting additional data, and the Avaya SENTRY and BETA 80 example we demonstrated last year at NENA and APCO proved the “over-the-top model” we presented to the FCC in 2012. Rave also built an “over-the-top” approach, providing a simple way to “crowdsource” any additional data, validate it through a public safety approval workflow, and then keep the data updated. The data ultimately expires or is “aged” out when it becomes no longer current.  Existing NENA i3-compliant interfaces allow any application to ingest the data, as well as responders and on-site personnel directly. In fact, Rave’s mobile interface provides that exact functionality.  Indoor location accuracy continues to improve, and National Clearing House NG9-1-1 repositories from RapidSOS are emerging. There is now a huge need to collect and maintain floor plans as a key component of making the improved location information actionable for responders.

Improving safety, whether at a school or in a hotel room, is never a one-size-fits-all approach, but we owe it to the public we serve to identify ways to improve collaboration and communication around incidents that occur with alarming and increasing, frequency.

 

For an AUDIO PODCAST version of this Blog [ LISTEN HERE ]

As we wind our way through hurricane season this year, and this Saturday marks the beginning of National Preparedness Month, we need to think about citizen safety and education. If there’s one thing that gets under my skin, it’s the manipulation of people’s feelings under the guise and banner of “Public Safety.” I believe that many take advantage of the limited knowledge that constituents have around public safety networks, and capabilities.

Why does this happen? Simple, unrealistic television. While I know that I sound my age when I say something like that, it is true. Our environment today is a visual one. Since Stanley Kubrick’s “2001: A Space Odyssey”, Hollywood has taught us that if we can dream it, it will come, although it may take a few years. Instead of HAL, we have Siri, Alexa, and a whole host of smaller players that we interact with on a daily basis. Quite often, it surprises me at the length and amount of energy we expel trying to do a simple task. Just this week, a colleague of mine was having trouble with her vehicle automation turning on the air conditioning. She diligently tried six or seven times, and I’m thinking, “you know, you can just press that button right there?” Despite whatever the problem was, she was persistent, and the air conditioning came on, and I could feel her sense of pride, proving that she had the ability of mastering technology.

Lisa is a smart person. She knew she could figure it out, and she wasn’t going to give up without a fight. Lisa is a smart person. She understands technology and knew that something was wrong and that she had to correct. Lisa is a smart person, and she was determined to show off her technical prowess to her colleague that she rarely gets to see in person. For myself, I already know Lisa’s a smart person, so I sat back and mentally critiqued her troubleshooting skills. Guess what? She passed.

That’s a long way to go to make a point that’s critical, and the purpose of this blog. While Lisa was smart and determination drove her to correct her temporary issue, when it comes to legislation and budgets and funding in the actual legislative process in a state, most people (myself included) never really paid attention to that day in high school. Also, we tend to sit back and let things happen. The risk that we run by doing that is that we end up with situations like the state of New Jersey diverting $2 billion over 14 years, which works out to be just over $391,000 EACH AND EVERY DAY. We also end up with situations as we have in California that just recently emerged. The California SETNA fund (State Emergency Telephone Number Account), is a fund based on collecting a small amount from each telephone number bill. That collection occurs on landline devices only and has been reduced by almost 40% over the last 10 years due to the migration of landlines to cellular phones, and voice over IP telephones.

To correct the issue, Gov. Jerry Brown has a plan to bring together State Bill 870 and Assembly Bill 1836, pending approval from two-thirds of the California legislature. This new bill establishes a flat rate on every access line that can utilize the 911 system, and provide for a monthly collection of between $0.20 to $0.80 per line. The fee would begin January 1 of 2019 and is expected to raise $138 million during its first year of operation. Not only would this money support the legacy network, but it would fund Next Generation 911 services for the state.

While this plan does have broad support, some naysayers are holding up this critical legislation in the legal process. They refer to a $9 billion surplus that the state currently has, and wants the money to come from there. While that seems like a plausible solution, the Gov.’s Office of Emergency Services issued a statement stating that “there is a significant danger in tying the future of the 911 system to a budget surplus that we have in 2018 and 2019. The legislation will last for another 25 years. What if we are flush this year and not next year?”

It’s a great argument. However, instead of arguing about “who gets the surplus?”, Why not figure out what’s wrong with the budget that gave you $9 billion in surplus, to start? Maybe a more palatable solution would be to pass a bill that would perpetually fund a 911 network and enhancements. These new laws need to modernize where and how we collect fees from all access lines. Finally, if you feel it is worthy, make a one-time cash donation from your “surplus,” and kick this project off to save lives, and therefore making a difference. While you’re doing that, you can get rid of your current accountant, who doesn’t have the ability budget well, in return for one that does.

While this may seem to be a California problem, it is not. Every citizen, every constituent, every voter needs to express their opinion. It’s how the system works, and it’s how things get changed but go awry. It is not by any means, a perfect world. However, we can do a better job of managing our life-safety services.

To tie this back around with the original paragraph of this blog, while you’re thinking about preparedness for September, make sure you have an emergency services network in place that can respond to your request for help. It seems that may be pretty high on the list of “being prepared.”

 

An AUDIO version of this blog is available on http://Avaya.com/APN

Like any other Sunday morning when I’m not traveling, I park myself at my local luncheonette; grab my big Avaya mug from the wall behind the counter, and stick on my headphones. I Then immerse myself into the weeks technology tweets and blogs from the people I respect the most in the industry.

Evan Kirstel is one individual that never ceases to amaze me, either with inspiring tweets of his own, or something worthy of his RT, and likely worth reading. Today, a particular article he posted caught my eye. It appears that a D-Link DNS hack has become problematic where Hackers modify the router DNS to their own nefarious DNS look-alike. By doing this, any requests made for the IP address of someone’s bank, for example, would return a spoofed address of the hackers look alike bank site. Unsuspecting users would then login with their normal credentials, which would be collected by the hackers and then used later to drain accounts.

The security exploit here, is that the person is relying on information from one location, the DNS entry in the D-Link Router. it assumes that the IP Address of the DNS server provisioned is valid and authenticated, which in fact is not. This immediately struck me as a classic use case for blockchain, and an excellent way to explain it to those who may not understand.

As it was explained to me, blockchain is an architecture that stores data in multiple locations (or blocks) across the Internet. The actual data in the block itself is irrelevant. In fact, it is the container of that data that is tagged with an identifier unique to that block of data. In addition, the identifier is changed each time the contents of the container is modified, and this identifier is replicated everywhere the data is stored across the internet. When a user retrieves a specific block of data, they can then compare the identifier of the container with the other containers of the information. Should there be a mismatch in the identifier, it becomes immediately apparent that the data is not current, or valid. So, while it may be possible, or even easy to hack a particular container, replicating that to every other instance of the container would be nearly impossible. The more secure the data needs to be, the more container instances would exist, making it near impossible without quantum computing power and many say even that would be questionable.

This makes the value proposition here quite simple. If I’m going to hack your data, I need to hack every instance of your data, or you will know that the proverbial “seal has been broken”. It would be clear to all that the data has been tampered with without actually seeing the data, which ensures privacy. Knowing this, let’s go back to our DNS hack example.

When your router establishes an online connection, the router obtains its IP address, subnet mask, default Gateway, and DNS server information via DHCP. To ensure that DNS has not been hacked and spoofed, with a nefarious spoofed server address, it would be a simple matter of inspecting the blockchain identifier on the DNS data coming in, and also when the contents have changed. This will confirm if the blockchain address is valid as compared to other known data sources.

If a hacker managed to attack your router and modify the DNS IP address entry, it would be immediately known and could be flagged as an insecure source of data. That being said, please remember this was intended to be just an example to illustrate how blockchain could be used in a very simple environment. As you evaluate new infrastructure and architecture, don’t ignore the block chain value proposition built within products. Understand what they have, where the manufacturer is going from a roadmap perspective, and what could be used to lock down your data in this ever-changing, and fast-moving Internet of Everything.

To bring this full circle back into my Public Safety practice, Next Generation 911 networks will be chock-full of data and information from various sources. Protecting our critical life safety systems on the backend will be a challenge. At the same time, we can no longer lock these systems away in the back room away from the data that’s needed to evaluate situational awareness that will ultimately save lives. I believe blockchain will play a significant role in the validation of that data, and the architecture that will allow good data to flow from the people who have it to the people who needed, while protecting those first responders from attacks by those looking to circumvent, and infect the system.

The lesson here is not only is guarded diligence. Understanding networks is critical in building our NG911 environment, but a specific eye on public safety security best practices is paramount. I’d love to hear your thoughts on the matter, and what you perceive as suggestions and fears for the future.