Redefining EMS success: A fire chief’s take on EMS performance metrics

Fire and EMS chiefs have historically assumed that the public knows our EMS systems are critical in providing timely and effective out-of-hospital medical care to communities – it seems obvious.

Traditionally, EMS performance has been evaluated based on response times, an objective metric easily understood by the public. We preached on the importance of minutes and talked about cardiac arrest survival, STEMI, stroke, trauma, sepsis and other critical emergencies.

Focusing primarily on response times, however, has led to increased costs, greater risk of response and transport crashes and an incomplete (and I would argue, poor) assessment of quality medicine.

Based on the latest release of a joint statement by 16 national and international EMS associations, their advice should resonate very loudly with fire and EMS chiefs and directors. They have been ringing the alarm for an overhaul of how we do business, including a comprehensive approach to EMS performance measurement encompassing a broader set of clinical, safety, experiential, equity, operational and financial metrics – and response time is not a key component.

|More: 16 EMS groups publish joint statement on EMS performance metrics

The limitations of response time as a sole metric

We designed our systems in the early 1970s to respond rapidly to life-threatening conditions, such as cardiac arrest. Remember, Johnny and Roy always drove fast, took chances and saved the day.

The data tells a different story – swift response is essential in few scenarios. Most 911 EMS calls do not necessitate a response time under 10 minutes [1]. That may be a hard truth to say out loud, but you know it, our people and the payers know it.

An overemphasis on response times has inadvertently increased our operational costs and the risk of vehicular accidents, while neglecting the crucial aspects of medicine delivery that equate to quality and effectiveness [1,2].

▶️| Matt Zavadsky on EMS metrics

Comprehensive EMS Performance Measures

The joint statement emphasizes the need for EMS systems to adopt a balanced performance metrics set that provides a more complete view of the quality of medicine being provided. These measures should be evidence-based and associated with improved patient outcomes and system performance. Response times are not one [3,4). The key areas we should be measuring (and focusing our efforts on) are:

Clinical measures

• Evaluate the appropriateness and quality of healthcare provided.
• Traditional key indicators [5, 6]:
  
  • Out-of-hospital cardiac arrest outcomes
  • STEMI
  • Stroke
  • Trauma
  • Hypoglycemia
  • Asthma/COPD
  • Seizures/status epilepticus
  • Invasive airway management
  • Special mental health crisis management

The actual reference for KPIs can be found at: National EMS Quality Alliance (NEMSQA) published measures, NEMSIS Public Dashboards, Cardiac Arrest Registry to Enhance Survival (CARES), AHA Mission Lifeline.

Safety measures

If we say our people are the priority, then demonstrate it. Literature reviews on the use of lights and sirens prove that minimizing their use can significantly reduce crash rates and improve safety without negatively impacting patient outcomes [2].

• Focus on the safety of patients, responders and the community.
• Suggested KPIs:
  
  • Percentage of responses and transports using lights and sirens
  • Crash rates per 100,000 miles
  • Job-related injuries and illnesses per 100,000 hours worked
  • Dispatch priority assignment reviews
  • Sick/injury on duty rates before and after modifications
  • EMS return rate after non-transport responses

Experiential measures

You don’t know unless you ask. Satisfaction is increasingly recognized as a critical component of healthcare quality, with significant implications for clinical outcomes and system performance [7, 8]. Often, we get comfortable in bureaucracies; we like the good news and settle in when things are comfortable. Don’t get comfortable, and don’t settle. Keep inquiring and build a growth mindset.

• Use patient satisfaction surveys and implement receiving ED team and ED clinician surveys to rate their satisfaction with your EMS services. Apply QR codes that link to surveys and put them everywhere – make it easy for the in-hospital teams, crews (put one on the dash), dispatch and ED techs (they matter) to send feedback.
• Key indicators – Data from:
  
  • Experience surveys
  • Your partner (transport or first response) surveys
  • Personnel engagement surveys
  • Employee turnover/retention rates
  • Dispatcher engagement surveys

Equity measures

Studies on healthcare equity emphasize the importance of providing consistent and unbiased care, highlighting disparities that can arise from a sole focus on response times [10].

• Ensure equitable care across different patient demographics and geographic areas. Ensure that your Community Risk Analysis includes a layer and perspective of your geographic population and its diversity.
• Focus on evaluating whether your EMS system provides fair and unbiased care to all segments of the community.

Operational measures

Research suggests that operational efficiency metrics can better identify system strengths and weaknesses, leading to more targeted improvements in EMS delivery [1].

• Closely examine the efficiency of EMS operations.
• Key indicators:
  
  • Produced unit hours vs. scheduled unit hours
  • Mission failure rates per 100,000 miles or less
  • Response times for proven high-acuity clinical responses based on paramedic impression and not dispatch
  • QA assessments of response prioritization reliability and dispatch appropriateness (look and listen closely to caller interrogation and utilize AI to transcribe for identification of keywords and further clinical analysis)

Financial measures

Financial performance indicators are crucial for sustaining EMS systems and ensuring that resources are allocated effectively to support high-quality clinical care [11].

• Know your budget and clearly understand your revenue and reimbursement. The economic efficiency of EMS services is something that is rarely taught in “chief school.” Building fires and fancy helmets always seem to win when prioritizing time in traditional organizations.
• Key indicators – costs and revenues per:
  
  • Staffed unit hour
  • Response
  • Patient contact
  • Transport
  • Dispatch staffing levels
• Sources to find more on EMS Financial KPIs: Academy of International Mobile Healthcare Integration (AIMHI) surveys and national data sources for comparable agencies.

Implementing a modernized approach

I urge fire and EMS chiefs and leaders to listen to our associations and this position statement.

We all need to emerge from the echo chamber where we hear naïve statements like, “You call, we haul,” “GEMT is endless,” “It’s the cost of doing business,” “We’re firefighters, EMS is just something we do in the meantime,” and so many others.

Perspectives like these are uneducated and fail to recognize the complexities and nuances involved in pre-hospital medicine. EMS is a profession, and we need to stop the tendency to oversimplify and make assumptions without considering relevant facts or data.

Those of us in fire-based EMS need to apply the same level of priority and effort we make at staying a traditional fire service to providing quality medicine.

What the effort actually looks like will vary for different communities and agencies. It could be the creation of innovative programs, such as mobile integrated healthcare, social workers, community paramedicine, alternative response models, 911 diversion and varied response dispositions to offer a broader array of services to patients [1].

The most important message for fire and EMS chief is to get out of comfortable. Read the position statement. Measure, change, measure again, change again and start the process all over. Change and save your EMS system before someone else does.

REFERENCES

1. Kupas, D. F., Zavadsky, M., Burton, B., et al. (2024). Joint Position Statement on EMS Performance Measures Beyond Response Times. [PDF Document].
2. Murray, B., & Kue, R. (2017). The use of emergency lights and sirens by ambulances and their effect on patient outcomes and public safety: A comprehensive review of the literature. Prehospital and Disaster Medicine, 32(2), 209-216.
3. Sasson, C., Rogers, M. A., Dahl, J., & Kellermann, A. L. (2010). Predictors of survival from out-of-hospital cardiac arrest: A systematic review and meta-analysis. Circulation: Cardiovascular Quality and Outcomes, 3(1), 63-81.
4. McNally, B., Robb, R., Mehta, M., Vellano, K., Valderrama, A. L., Yoon, P. W., Sasson, C., & Kellermann, A. L. (2011). Out-of-hospital cardiac arrest surveillance—Cardiac Arrest Registry to Enhance Survival (CARES), United States, October 1, 2005–December 31, 2010. MMWR Surveillance Summaries, 60(8), 1-19.
5. O’Gara, P. T., Kushner, F. G., Ascheim, D. D., Casey, D. E., Chung, M. K., de Lemos, J. A., ... & Zhao, D. X. (2013). 2013 ACCF/AHA guideline for the management of ST-elevation myocardial infarction: A report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. Circulation, 127(4), e362-e425.
6. Terkelsen, C. J., Sørensen, J. T., Maeng, M., Jensen, L. O., Tilsted, H. H., Trautner, S., ... & Andersen, H. R. (2010). System delay and mortality among patients with STEMI treated with primary percutaneous coronary intervention. JAMA, 304(7), 763-771.
7. Cleary, P. D. (1999). The increasing importance of patient surveys. British Medical Journal, 319(7212), 720-721.
8. Manary, M. P., Boulding, W., Staelin, R., & Glickman, S. W. (2013). The patient experience and health outcomes. New England Journal of Medicine, 368(3), 201-203.
9. Blackwell, T. H., & Kaufman, J. S. (2002). Response time effectiveness: Comparison of response time and survival in an urban emergency medical services.
10. Whitehead, M., & Dahlgren, G. (2006). Concepts and principles for tackling social inequities in health: Levelling up Part 1. World Health Organization.
11. Jacob, R., McKinney, N., & Müller, D. (2019). Cost-effectiveness of emergency medical services. Journal of Health Economics, 64, 1-14.
12. Cardiac Arrest Registry to Enhance Survival (CARES). (n.d.). [Online Resource].
13. Clawson, J. J., Martin, R. L., & Hauert, S. A. (1994). Protocols vs. guidelines for emergency medical dispatch. Annals of Emergency Medicine, 23(6), 1274-1278.
14. National EMS Quality Alliance (NEMSQA). (n.d.). EMS performance measures. [Online Resource].
15. National EMS Quality Alliance (NEMSQA). (n.d.). EMS Performance Measures. [Online Resource].
16. Cardiac Arrest Registry to Enhance Survival (CARES). (n.d.). [Online Resource].
17. AHA Mission Lifeline. (n.d.). [Online Resource].
18. Clawson, J. J., Martin, R. L., & Hauert, S. A. (1994). Protocols vs. guidelines for emergency medical dispatch. Annals of Emergency Medicine, 23(6), 1274-1278.
19. Blackwell, T. H., & Kaufman, J. S. (2002). Response time effectiveness: Comparison of response time and survival in an urban emergency medical services