Since 2023, Sutter has been addressing the provider burnout epidemic and the burdensome administrative tasks detracting from the time and focus clinicians dedicate to patients. Dr. Veena Jones, Chief Medial Information Officer, Sutter Health explains "by leveraging generative AI, we’re reducing these burdens, allowing clinicians to focus on meaningful, personalized interactions with their patients while enhancing the overall care experience and the clinicians’ joy of work. It’s truly a win-win for both providers and patients.”

Sutter Health, a non-profit integrated delivery health system providing comprehensive care to nearly 3.5 million patients throughout California, is one of the early health systems thoughtfully exploring ways to test AI-driven solutions to support patient and provider needs.  With a focus on responsible implementation, Sutter is committed to leveraging AI in a way that enhances care while maintaining the highest standards of security and trust.

Specifically, Sutter has invested in AI & GenAI for two key initiatives as “tools for our providers tool kits”:

     1) Supporting Patient Messaging within Epic, Augmented Response Technology (ART) – The ART tool analyzes patient messages, gathers relevant information and drafts a response for the clinician to review, edit and approve, before sending to patients.  Integrated within Epic, ART uses a large language model (LLM) to generate a contextually relevant response for clinician review.

       In Q4 2023, Sutter was one of the first handful of health systems that began piloting this tool.“Our early work with Epic and other leading health systems to pilot the ART tool was instrumental in advancing its development,” said Dr. Jones. “Since October 2023, ART has been in production within the Sutter EHR, following a collaborative effort with primary care clinician champions and patient advisors to help validate and refine its capabilities.”

       Sutter started testing ART with about 70 physicians and advanced practice clinicians to gather feedback on messages generated from the LLM and adjust the prompts before the March 2024 launch. Beyond clinician feedback, the team collaborated with patients for feedback on GenAI generated message response.

      "It was vital to include perspectives from patients in our Patient Family Advisory Council (PFAC) program," said Dr. Jones. "By reviewing message responses with patients, we gained invaluable insights—ensuring that the technology supports clear, compassionate, and meaningful communication. Patients are at the heart of our mission and it’s essential to align AI advancements with the human-centered care we strive to deliver.”

       In March 2024, Sutter launched ART to all primary care clinicians in Internal Medicine and Family Medicine, since they were on the receiving end of the most patient inbox messages. Soon after, ART was made available to all pediatricians and then was expanded to OB/Gyn and geriatric clinicians across the organization.

       Sutter recognized the need to adjust the LLM prompt to ensure relevance and benefit with each specialty rollout, working closely to test ART before making it more widely available. By the end of 2024, all Sutter clinicians receiving patient messages in their in basket had ART-generated draft replies available for their use.  

     AI PATIENT MESSAGING SUCCESS:

     “We started the ART project with the goal of saving providers time when responding to patient emails. What we heard overwhelmingly was that providers want to keep using it because it significantly reduces their cognitive burden”, shares Laura Wilt, Sutter’s Chief Digital Officer. “Think about the level of effort required for a doctor to create a patient response email from scratch versus simply reviewing and editing a pre-drafted message.”

·        To date, 3,000+ Sutter clinicians have used ART to assist with responding to medical advice requests from patients.

·        With over 217,000 messages sent, feedback from Sutter clinicians has been invaluable in refining the prompt engineering process to enhance the quality of draft responses.

·        Clinicians are reporting an average time savings of up to 20 percent when using ART to help manage their patient communications, along with a noticeable reduction in cognitive load and mental fatigue.

Qualitative Provider Feedback:

“I like the empathy and the AI component that reviews the chart and gathers information like last labs, or pending labs. That is very helpful.”

“I like that the responses are often more thorough and considerate than I would have time to write.”

“Includes empathy statements and acknowledgement of patient's symptoms. Really good for quick and straightforward questions.”

“Appreciate the ART-generated responses to long patient messages. Reduces mental workload by having a template initially started.”

2) Ambient Listening for Visit & Follow- up with Patient Friendly Summaries. Sutter has been working closely with Abridge which has a generative artificial intelligence (AI) platform for clinical documentation. It records and transcribes a visit and generates components of the clinical note. In addition it creates a patient-friendly summary of the visit which the provider can paste into their After Visit Summary.  

To date, most clinicians using Abridge have seen a reduction in documentation time and cognitive burden associated with their work. The ambient listening technology allows for more personal connection and focus on the patient interaction, rather than on the computer, thus strengthening the care relationship.    

“Another benefit we are seeing with Abridge’s ambient listening technology to personalize care is enabling providers to conduct the visit in the patient’s native language, while the English documentation is created in real-time. We have patients and clinicians who speak a variety of languages, from Spanish to Cantonese. The mental load is high for these clinicians having to translate everything they discuss into English for the note. Abridge takes care of the translation and puts the note directly into the EMR”, explains Wilt.

From April- December, Sutter enabled 1,000+ ambulatory physicians and advanced pratice clincian users with the ambient technology. Sutter learned about the importance of physicians being able to directly access Abridge through Epic’s Haiku app, having it fully integrated to make it easier to have their work all in one place. Wilt adds “we took a thoughtful approach to implementation, aiming for a smooth adoption. Our Digital Academy Team played a key role in supporting the rollout and ensuring physicians had the resources they needed. It went smoothly, with minimal training needed— just a brief eLearning video to watch. From a physician adoption perspective, it took off like a rocket ship… maybe because it takes tasks away and lets providers focus more on connecting with patients.”

AI AMBIENT LISTENING SUCCESS:

“Again, we initially focused on time savings, and for many clinicians it’s a significant time saver. But we’ve found that —it also helps reduce cognitive burden”, Wilt adds.

      Qualitative Provider Feedback: 

“Abridge has been a life-changing experience. It alleviates 50% of the charting work for me,” said Alice Woo, M.D., a plastic surgeon with Sutter West Bay Medical Group. “[Now] I feel like my conversations with patients are much more intimate and therapeutic.”

“Abridge has changed how I feel at the end of the day,“ said Kevin Chen, M.D., a family medicine physician with Palo Alto Foundation Medical Group. “If you have balance in your life, you can do other things. It makes you a better doctor when you’re in the clinic seeing patients—you’re better centered.”

EXPANDING AI USE CASES:

“We see the patient message response/ART in Epic and Abridge Ambient listening platform as examples of how generative AI technology can help Sutter personalize medicine”, Wilt shares. 

Patient Message in EPIC/ART. Sutter is interested in moving beyond the patient generated messages into other message types such as “results” (i.e. normal results) and “medication” (i.e. medication refill requests). Sutter is also planning a pilot with Epic for another feature to help further streamline physician workflows with “Chart Summarization”, which provides a summary of key patient information within their electronic medical record. 

Ambient Listening/Abridge: Sutter is extending use of the ambient listening tool in a pilot with emergency and inpatient physicians across all Sutter hospitals. They see opportunities with other types of clinical staff, such as with nurses and physical therapists, and look forward to exploring these as the technology evolves. 

Sutter is excited about the continued evolution of Abridge which will "go beyond capturing the conversation and generating a note, to queuing up clinical orders that were discussed. Wilt explains, “so if your doctor mentions getting an Xray, the order would be put in for the physician to just accept, instead of having to type in the order.” 

“AI is all about continuous improvement—what was implemented a few months ago has already evolved, and it will continue to improve over time," said Dr. Jones. She encourages Sutter teams and other health systems to approach these tools with curiosity. "When implemented thoughtfully, AI has the potential to not only reduce administrative burdens but also strengthen the connection between providers and patients, enabling more meaningful care experiences."

MY TAKE: AI INNOVATION BEST PRACTICES

Sutter has invested in the right practices and processes to drive success:

1. Collaborated closely and early with vendors (e.g. Epic, Abridge) to define and pilot AI and GenAI capabilities.
2. Co-created with key clinical and patient stakeholders to ensure a strong experience.
3. Integrated with exisiting workflows spanning development, training, care delviery and support. 
4. Focused on both qualitative and quantitative success measures including the voice of the clinician and patient (e.g. feedback) and user trial/usage/traction. 

During the HIMSS AI for Healthcare Forum in Boston, Dr. Eve Cunningham, MD, MBA, GVP, Chief of Virtual Care and Digital Health at Providence Health presented a few AI use cases including an impactful example which empowers PCPs for better care decisions.

Today primary care doctors are pressured to address patient health concerns in an average of 18 minutes with an average of 220,000 data points in the patient’s electronic medical record. With over one in 3 patients referred to a specialist each year, PCPs are tasked with diagnosing the health issue to and making decisions about the need for a specialist.

Barriers to Specialist Care Decision

With vast amounts of patient data that is not clinically organized around context within the patient’s chart, PCPs must spend time pulling together and reviewing relevant information to support the patient’s health problem. Then the PCP must determine if a specialist is needed, and if so which type of specialist.

Given the shortage of primary care doctors and specialists, optimization of the referral process is critical to access to care. However, the referral process today is both inefficient and ineffective causing problems:

• Patient did not need to go to a specialist. (Note: About 20% of patients referred to specialists do not need to see a specialist at all)
• Patient is sent to wrong specialist and/or at inappropriate level of urgency.
• Patient arrives to specialist without any prereferral work-up such as lab, imaging and first-line therapy for a condition.
• Patient has delayed care until there is a specialist appointment

Accelerating Primary Care Referrals

The September issue of NEJM Catalyst’s article on “A Physician-Created Platform to Speed Clinical Decision-Making and Referral Workflow” describes the problem and process that Providence Health has undertaken to conceptualize and build their MedPearl Platform.

“As primary care clinicians struggle to access the expanding base of specialized knowledge and experiences, they attempt to refer patients to specialists, often without the advantage of usable tools at the point of care.”

Developing this needed platform entailed using a “Human- Centered Design approach with a Jobs to Be Done framework, working closely with our clinicians and UX designers”, explains Dr. Cunningham. “We began by asking clinicians how this platform needed to work and look, and got input from 270 clinicians in the design process, which was essential for gaining buy-in”.

MedPearl Platform Built by & for Clinical Care Decisions

Based on clinician’s requirements, Providence built MedPearl as an application within their Epic electronic medical record (EMR) to integrate a digital assistant with clinical knowledge to support  primary and urgent care clinicians. MedPearl has “concise, human-authored algorithmic primary and subspecialty care next best actions on over 700 conditions”.

Clinicians utilize MedPearl’s AI algorithms to pull up and review essential clinical information alongside contextualized patient data to see next best actions with what has/hasn’t been done for the patient, empowering clinicians to optimize the “work up” while reducing redundant diagnostics.

Since it can take weeks to months for the patient to get in to see the specialist, the MedPearl platform enables PCPs and urgent care providers to take care of the diagnostics in the interim to optimize the care referral decision. With MedPearl, clinicians can see the testing stream, make sure tests do not need a referral, find the most appropriate specialists and triage the urgency of the specialist appointment based on patient history and diagnostics. Think about what this means for rural health where accessing a specialist is even more challenging.

Providence has designed MedPearl with icons to reduce the clinicians cognitive reading load and provides links to guidelines and QR codes to scan for evidence -based applications.

Clinician Experience with MedPearl

Providence Health has deployed MedPearl across their health system which is available in Epic’s Marketplace.

Dr Cunningham as MedPearl founder comments “Our clinicians tell us that with MedPearl they are spending less time in the EMR, fewer outside work hours and are more productive. To date, we have 7,000 clinician users, making 30,000 searches per month on the MedPearl platform”.

During the initial pilot with 216 participants conducting 14,000 searches, clinicians reported the clinical content helped with clinical decision support: they did not need to refer the patient 20% of time, improved care plan and work-up 72% of the time and changed referral specialty and/or level of urgency 20% of the time.  

With MedPearl, clinicians are also realizing better resource efficiency. “We had a challenge with some of our more newly onboarding clinicians overwhelming our Medical Directors with clinical questions and curbside phone calls. The call volume   decreased significantly when these clinicians were advised to reference MedPearl first for answers”, shares Dr. Cunningham.

Clinicians often walk through the MedPearl tool with their patients. They review educational content together, discuss the algorithms and use the decision support tools. 

Feedback from Clinicans Using MedPearl

After using the MedPearl tool, clinicians commented about the ease of use, quick access to the most essential patient data and benefits of leveraging this patient data for the physician, practice and patient.

   “MedPearl provides concise and focused clinical evidence -based summaries, making it easier to quickly access the most relevant information. Its streamlined format saves time and effort, ensuring health professionals can find information without being overwhelmed by unnecessary details”. Medical Director Urgent Care 

   “The cognitive offloading and reassurance that knowing the Prov approved standard of care has been helpful for me, and I love being able to share it with newer clinicians”. Urgent Care Physician

   “The Headache algorithm – perfect example of how to reduce variation in our urgent care practice.” Urgent Care Physician

   “With the demand for neurology consultation greatly exceeding the supply of neurologists for the foreseeable future, MedPearl improves access when it has never been more vital.” Chief of Neurology

   “MedPearl is a point of care tool we need to augment our clinical acumen and to help drive uniformity of patient care path the best -practice level. This allows primary care physicians to provide more care value, improving diagnostic testing, treatment, and referrals and in opening more specialist access.”  Medical Group Director

   “Recommendations re: referrals and specific imaging with and without contrast very helpful. Truly has improved physician satisfaction and flow for the day”.  Family Medicine Physician 

   “Contextualization is pretty cool. Makes Pearl useful in multiple ways.. Quick way to see all relevant data about a work up for a particular topic.” Internal Medicine Specialist

   “Love the patient experience, engagement and shared decision-making components of this tool.”  Practice Senior Director of Operations

   “I love the headache algorithm. During a recent appointment, I had a patient point to each symptom that applied to her.”  Physician

Future Direction of MedPearl for Health Systems

Dr. Cunningham describes Providence Health’s approach with “feedback and iteration as a core principle by which we drive our MedPearl features roadmap and design. We also push our end users to think beyond the workflow of today, and envision workflow of tomorrow.”

The MedPearl team continues to collaborate with users to gain insight into future enhancements. Specifically, they are focused on these platform innovations:

1. "Expanding contextualization beyond labs, imaging and procedures to further enrich the context aware data aggregation experience"

2. "Adding documentation support features to expedite documentation workflow"

3. "Further refine home page to create a more dynamic experience to delight end users"

4. “Since we believe all workflows will converge on ambient technology, we are preparing ourselves to integrate into ambient workflows. We don’t plan to be the ambient provider, rather we envision ourselves as a clinical intelligence plug-in to further enrich that experience.”

Dr. Cunningham confirms plans to begin “deploying MedPearl to other health systems using EPIC and then Oracle EMRs. We see ourselves as ‘headless’ clinical decision support system that is EMR agnostic and application agnostic. We want to permeate and spread the goodness of MedPearl in a seamless fashion, and meet clinicians and patients where they are“, Dr. Cunningham concludes. 

During the recent Home & Hybrid Care Virtual Summit 2024, St Lukes Health shared how they have expanded virtual access to support patient care journeys over the past three plus years. It is refreshing to see how St Lukes views virtual as a big front door to their care network, how they have personalized the entrance and experience across more than a half dozen patient programs and how they have invested in an integrated data platform and tools for better care experiences.

Like other health systems, Idaho- based St Lukes Health is addressing today’s healthcare headwinds including staffing challenges, provider burnout, the consumers increased expectations for their healthcare experience, all within the regulatory and reimbursement landscape.

In September 2021, St Lukes Health launched a telehealth program to patients with one of seven low acuity conditions.  Since then, their virtual program has expanded to provide patients with immediate care access until they can schedule and see their providers, which may take up to a few months.

“Our team value is that we (virtual care) will be the quickest access to a provider”, explains Abby Losinski, MHA, Director of Telehealth at St Lukes. Abby’s group is part of the Consumer access and experience team.

St Luke Virtual & Hybrid Model

Since late 2021, St Lukes has developed needed processes and infrastructure to successfully expand their virtual program with a centralized virtual support team, virtual command center platform to address incoming patient demand and virtual care patient data integration to support care collaboration and a superior patient experience.

While many health systems launch separate virtual programs to support urgent care, mental health,  and chronic conditions, St Lukes centralized team supports many patient populations with virtual care access. They view this virtual interaction as an opportunity to begin a patient relationship.

Several patients targeted by St Lukes for their virtual care program:

• New Patients (December 2021) St Lukes promoted virtual services to welcome patients without a doctor but with an immediate need. During the virtual visit, patients can refill a prescription, get scheduling help and access care support until their appointment. Patients needing a higher level of care are quickly connected with a clinician. 

• Urgent Care (May 2022) Patients at St Lukes’ Urgent Care Centers with high wait times are offered a virtual appointment when they meet certain criteria. Patients use their mobile phones for the visit and are urged to stay nearby if point of care testing is required. This gives patients a chance to be seen when there are no available rooms. 

• Service Line Extension (January 2023) “We are an extension of any department where patients need same day access” (e.g.  primary care, urology, oncology, pediatrics) explains Losinski. “We work with the Medical Directors to build clinical guidelines so that patients are treated the same way whether they are coming in to see a specialist or meeting with our virtual clinician”.  Losinski is proud that St Lukes “can offer multi-specialty care from one centralized team”. 

• Maternity/New Pregnancy (April 2023) After learning about their pregnancy, patients often have questions and concerns. Instead of waiting for their first appointment which may take 8-10 weeks, patients can meet with a clinician virtually and set up notifications in MyChart to immediately receive information about their maternity care and growing baby. During the virtual visit, patients can get scheduling help and care can be escalated if required. 

• Home & Community Access (Sept – December 2023) St Lukes has set up “virtual hubs” enabling patients to connect virtually using digital health tools from a partnership with TytoCare. “We wanted to expand what we can treat virtually because we have new tools. If we can’t see into the ear, we can’t treat it” and the revenue is lost, Losinski explains. St Lukes now offers TytoCare Home devices (e.g. otoscope, stethoscope, throat camera, skin camera) to patients at home and TytoCare Clinic to patients at schools, libraries and workplaces, to capture data for the virtual care team decision support.

St Lukes Virtual Care Access Experience Success 

Their virtual program success is achieved through continuous focus on the patient experience. St Lukes has designed their virtual experience from the outside- in and offers patients the bridge into their health system care network.

Here are several success elements that I have noticed to deliver care continuity:

Designed Patient Experience to fit care scenario. Newly pregnant patients are set on the Maternity path, supported by the virtual team with education, testing and virtual visits until their OB appointment. Service Line Extension patients such as oncology may have access to digital tools for remote monitoring with condition education and support until their specialty appointment.

Partnered for Patient Care Coordination. St Lukes’ Consumer access & experience team has built partnerships with different service lines and settings across their network. Patients accessing virtual care at St Lukes needing more care are escalated to these clinical partners within the St Lukes network. St Lukes ensures patient care quality is maintained by using the same clinical guidelines for virtual and in- person care.

Built Integrated Patient Data & Tech Infrastructure. “St Lukes has decided that virtual care can not be this thing that lives outside of the health system”, Losinski emphasizes. St Lukes has invested to integrate and augment patient data (TytoCare devices) enabling clinicians to access information from all clinical interactions and from patients from beyond the walls to deliver better patient care and experiences.

St Lukes Virtual Care Feedback

Through marketing research, St Lukes has learned: 

Consumers: The “brand” offering virtual care services really matters. Consumers like that the doctors conducting the virtual visits work for St Lukes and live in the community and appreciate that their PCP can view their virtual visit notes.  

Patients: After participating in St Lukes’ virtual services, patients rated the experience an equivalent of 4.9 stars (out of 5). “Thank you for being there when no one else was” shares a patient.

Providers: Clinicians delivering virtual services have a strong provider experience; “appreciate the increased location and schedule flexibility”, “being in the moment for patients” and “getting to work from home”.

St Lukes Virtual Care Direction

Earlier this year, St Lukes launched eVisits, an asynchronous virtual care option. For $29/visit, patients can message a clinician about one of seven symptoms. eVisits are viewed as a virtual care “entry point which affordable and convenient for patients”.

St Lukes will continue expand virtual care access for their patients. “Currently, we are 8am-8pm but we are looking to extend our hours to 24x7” Losinski shared.

It is impressive to see how St Lukes brings patients into their care system through a larger virtual care front door, leverages care partnerships and protocols for consistent patient care quality and empowers care collaboration with integrated data and tools. St Lukes’ virtual care investments and vision enable them to effectively address staffing and consumer expectation challenges while delivering a superior patient and provider care experience.

In October 2022, Providence Saint John's Health Center invested in Project Ronin, an AI driven cancer intelligence platform to empower cancer patients and their clinical teams to manage treatment symptoms, care communications and clinical decision support. Their goal is to reduce ED visits and deliver better patient care and outcomes.

According to the American Cancer Society’s 2023 Cancer Statistics report, cancer remains the second highest cause of death after heart disease. This is an estimated 2 million newly diagnosed cancer cases resulting in about 610,000 deaths.

Dr. Martin, MD, CMO of Providence Saint John’s believes that providing a new “space where cancer people can engage with their physicians has some very real, tangible benefits.”  The digital tool is designed to support “patients with less-critical symptoms, which has the potential to reduce emergency department visits and improve clinical efficiency, which means a more efficient utilization of resources. Of course, we always advocate patients to come to the emergency department if they believe they are experiencing a medical emergency.”

Cancer Care Collaboration 

Successful treatment of cancer is multifaceted, requires a multi-disciplinary care team, entails managing different data sets with continuous capture and monitoring of data, and has data complexity given the non-linear nature of the disease (i.e. diagnosis, treatment, remission, recurrence, treatment, survivorship/death).

Patients struggle to manage their cancer, both mentally and physically. Patient treatment symptoms can be severe. Questions come up between treatments but care teams are extremely busy. Patients need to remain connected to care teams to stay engaged, better manage, and communicate side effects.

Providers try to navigate their inefficient clinical workflows, address the administrative burden due to documentation requirements, and deal with workforce shortages. Today, providers spend 20-40 minutes finding patient info to prepare for patient visits, sometimes missing key data.  Providers are overwhelmed to make needed decisions for cancer patients at the point of care.

AI Cancer Platform Delivers Care & Decision Support

The Ronin Cancer Platform enables patients to “take control of their care in a way that wasn't possible before and feel better connected to their care teams,” explains Dr. Neil Martin, MD, CMO of Providence Southern California Clinical Institutes and Executive Director of Saint John's Cancer Institute.  Clinicians utilize this cancer intelligence platform’s predictive analytics and clinical dashboard to define treatment pathways and support more informed care decisions.

How does this work? The Ronin cancer platform takes structured and unstructured data from different EHR silos, and then cleans, calibrates, contextualizes this information to present care insights. With predictive analytics, providers can identify at-risk patients and see a complete real-time view of the care journey for immediate actions.  

AI Data- Driven Cancer Platform: Patient Experience

Patient Patricia (patient persona), 74 years old, is newly diagnosed with breast cancer. While meeting with her new care team, Patricia learns about new resources including the free Ronin Symptom Monitoring App, to educate and support her as she navigates her cancer path. During her next appointment, Patricia notices the poster in her doctor’s office prompting her to download this patient app.

   Patient Symptom Monitoring App Poster:

We want to support you through your treatment. In between your visits, please let us know how you're feeling by using the Ronin Symptom Monitoring Application. This will help us catch any side effects that you may experience. Studies show that symptom monitoring can ...

• Help people live longer and improve their quality of life

• Help people stay on treatment longer

• Lower visits to the emergency room and reduce hospital stay

Patricia downloads and begins using the Symptom Monitoring app which captures her Patient Reported Outcomes (PROs), enables her to track and manage side effects for treatment tolerance, displays her symptom history and provides her with personalized education based on what she is reporting. Within the app, Patricia selects from a list of symptoms and then sees a message that her symptoms “are concerning”, that her care team is monitoring this and will reach out to her within 24 hours.

AI Data-Driven Cancer Platform: Provider Experience

Patricia’s care team receives a “symptom alert” through the EHR based on the information that she has entered. Her symptom information has required a response (e.g. new symptom, moderate or severe symptom). The Nurse Navigator reaches out electronically or telephonically to check in on Patricia to understand more about her symptoms.

When her Oncologist Dr. Davis initially set up Patricia’s personalized care pathway on the platform, he indicated her specific disease type, which triggered patient reported outcomes surveys and at-risk surveys which captures data and intelligently prioritizes for the care team to take quick action to course correct and preempt adverse events.

While preparing for an upcoming office visit with Patricia, Dr. Davis uses the Ronin Platform to view a longitudinal timeline of her cancer journey, filled with real-time data (e.g. labs, images, pathology, treatment, surgery notes, past encounters) to make informed, proactive and personalized treatment decisions.  Following the visit, Dr. Davis (or the clinical staff) pushes educational content to help Patricia to navigate her new symptoms or cancer stages.  

AI Driven Care Experience Value Proposition

At Providence Saint John's Health Center, both patients and providers are finding value using the AI Ronin Platform, including clinical efficiencies, better care and outcomes and higher satisfaction. Here are some comments from the patient and provider users:

 Patient Experience:

 “Ronin provided confidence someone was monitoring me.” 

 “Ronin made me feel like my care team was with me, even at home.”

 “I prefer not having to call the hospital. (With app) I am able to record side-effects right away as they are happening”. 

 “It’s (app) so easy to open up and write how I’m feeling…It’s a great reminder to me that even though I may think I’m fine, I need to pay attention to my symptoms.”

Provider/Staff Experience:

  “We were able to catch an immune- related toxicity early for a patient of mine. Without Ronin, I would not have caught it”.  Medical Oncologist

  “We want to make sure the patients are having the best quality of life. With Ronin, we can check in on them more frequently.”  Nurse Navigator

  “From the treatment team’s standpoint, it’s made it easy to follow patients. It’s a great way to stay hands-on.”  Nurse Navigator

  “I had a patient straight up tell me ‘If it wasn’t for the app, I would be calling you guys every day'.” Medical Oncologist

   “Visually, the Timeline tab makes it much easier to get up to speed on a patient. Especially compared to reading my colleague’s last notes.”  Medical Oncologist

 Success Metrics

In addition to gathering qualitative comments about the patient and provider/staff experience, Providence Saint John’s tracks quantitative success metrics for the AI driven Cancer Program. Some key metrics include:

● Reduced unnecessary ED admissions by >20% (early findings)

● Improved efficiencies with 35% decreased patient call volume and 32% patient-initiated messages.

● Strong patient satisfaction & experience

• 84% see positive impact on care experience
• 90% say app is easy to use
• 88% are satisfied with solution

Future for Providence with Ronin Cancer Platform

Moving beyond the patient mobile app and care communication from phase one, Providence Saint John's is planning to provide clinical decision support with oncology and risk dashboards and the capability to view comparative patient analytics in phase two.

"We are proud to implement evidence-based care solutions for patients in our community," said Brad Bott, Executive Director of the Southern California Clinical Institutes. "Our partnership with Ronin will help improve cancer care delivery while empowering our staff to practice more efficiently. It's a win-win."

Ochsner Mobile O BarOver five years ago, Ochsner Health launched their O Bar (Apple genius-like concept) to support patients getting started with digital health tools. Today, Ochsner has nine physical O Bars located in the bottom floor of their health centers and one mobile O Bar.

Although any Ochsner patient can visit the O Bar to begin using a curated set of digital health apps and devices, patients who are invited to join a digital medicine program can go to the O Bar to get set up with selected digital tools to manage and monitor their health journey.  Digital medicine program participants have the option to have their digital tools sent via the mail, without going to the O Bar and can call their program tech support for any assistance. Pre-Covid, about 5-10% of patients chose to receive their digital health tools by mail. During the pandemic, it is mostly all mail.

Ochsner Hypertension Digital Medicine ProgramTo date, Ochsner Health is offering digital management initiatives for hypertension, diabetes, pregnancy and the latest COPD program. Patients do not need to have access to WIFI to participate, just a smartphone or tablet. Less than 5% of patients participating in the digital medicine program use the tablet and instead have their apps downloaded to their smart phone. 

“Our digital medicine programs are realizing 2-3x better outcomes rates than the standard of care”, explains Dr. Richard Milani, Chief clinical transformation officer and innovationOchsner Medical Director, Ochsner Health. “In order to understand how these programs were designed, it is important to take a step back and think about how we need to help patients manage their chronic condition(s). First, we need more frequent data to know at any point in time if their chronic disease is under control. Second we must make sure individuals are prescribed guideline-directed pharmacotherapy.  With the number of new medications coming onto the market and medical research about the profile of patients experiencing the best outcomes, we must be sure patients are having the best chance to achieve an optimal outcome. Finally, we need engage our patients on the “right behaviors” (e.g. nutrition, fitness, stress reduction).”

"We set up our digital medicine programs to be supported by a dedicated team who interacts with and manages the patient’s condition(s)”, shares Dr. Milani. “Their doctor invites the patient to join the program, but it is a digital medicine team who responds to the incoming data and alerts from the digital health tools.” Ochsner’s digital medicine team consists of a pharmacist/APP to help the patient with the “right guideline-directed medicine” and a health coach to provide guidance on lifestyle decisions using behavioral health science techniques. Patients that are on two of Ochsner’s digital medicine programs engage with the same pharmacist/APP and health coach, creating a holistic approach to patient care.

Patients access all of digital health tools in the digital medicine program area with their patient portal. A patient logs in to access patient education information (videos), communicate with her team by scheduling a phone call or sending asynchronous messages and view monthly reports which shows how she is doing, and progress made over time. The patient can also contact the digital medicine team for technical support for their digital tools, which is rare because these connected devices are easy to use for even less tech savvy patients.

Ochsner Connected Mom Pregnancy Digital Medicine ProgramIn addition to appropriate connected devices given to patients to capture and transmit key measures (e.g. diabetes/wireless glucometer, hypertension/ wireless blood pressure cuff, COPD/wireless inhaler and pregnancy/ wireless blood pressure cuff & wireless scale), patients receive texts to capture changes in condition (e.g. COPD severity level), track self- efficacy measures or to be notified of a health concern (e.g. warning about the poor air quality level). Patients have the option of connecting in and sending more information such as weight measures from their own digital scale or steps from their fitness tracker to share with the digital medicine team.

Program Success Measures & Expansion Plans

Ochsner has enrolled more than 15,000 patients across their digital medicine programs.

Over the past 5 years, Ochsner has received positive feedback from their digital management team (e.g. Pharmacist, Health Coach) and from patients in the program.

 “The role of a clinical pharmacist isn’t always to add more medicine. We work with each person to incorporate lifestyle changes and medications that are right for them. This includes stopping or decreasing medicine doses when lifestyle changes lead to improved health.”  -- Carrie, Clinical Pharmacist

“I work with individuals to make small, achievable goals that will not only improve their health, but ultimately improve the way they feel mentally and physically. This allows the patient to feel confident in themselves to make healthier choices in any situation.” – Christina, Professional Health Coach

“I like that it is private. I don’t have to take a blood pressure reading at a Walgreens or CVS. It’s encouraging to know that the lifestyle choices I’m making as well as my compliance to my drug regiment is having a positive effect.” – Alan, Digital Medicine Hypertension Program 

“For anyone who has doubts about joining the program, I would say step out and take the journey.” – Lance, Digital Medicine Diabetes Program

“You get a lot out of Ochsner Digital Medicine. You get a family who is by your side every step of the way.”  - Gaylan, Digital Medicine Hypertension Program

In addition to patient and staff feedback, Ochsner uses a set of quantitative measures to evaluate success. Dr. Milani is proud to share the Net Promoter Score of patients in the digital medicine program of 87.5, which indicates a high level of recommending the program to others.

Dr. Milani explains, “the key success measure is the reengineering of chronic disease care into a new model of care delivery. Our metrics of success are control measures for the disease (i.e. better blood pressure control, better diabetes control, etc.).”

Ochsner has plans to grow their digital medicine programs in 2021.  “We are expanding the population we currently serve and will be adding more disease categories (like lipid management and others). We look at the prevalence of disease burden and the opportunities for better control when deciding on new digital medicine programs,” Dr Milani concludes.