Czech financial savings financial institution Česká spořitelna, a division of Austria’s Erste Group, lately collaborated with AI answer builder DataSentics to discover the usage of GenAI in name facilities. Česká needed to enhance high quality management and optimize prices of their inbound name heart operations, which obtain round 2 million calls per 12 months. They selected the Databricks Knowledge Intelligence Platform to experiment with each inner and exterior AI fashions to evaluate the effectiveness of name heart brokers.
Exploring a High quality Management System for Buyer Assist
The decision heart workforce at Česká spořitelna needed to check a high quality management system powered by GenAI that might be certain that brokers adhere to scripted pointers throughout buyer interactions. A essential problem for Ceska was making certain constant agent communication for routine buyer inquiries. When clients name about account balances, brokers have to direct them to on-line banking options, a key enterprise requirement that drives digital adoption and operational effectivity. The assist workforce wanted a scalable solution to confirm agent compliance and keep communication requirements throughout 1000’s of buyer interactions. To realize this, the workforce started through the use of Whisper, a speech-to-text mannequin from OpenAI, to transcribe conversations precisely. The problem was to supply human-readable textual content that precisely represented spoken phrases utilized by name heart brokers with out distorting their that means. The transcriptions wanted to make logical sense and mirror the intent of the dialog precisely for additional evaluation.
Following the transcription, the workforce explored integrating each inner GPT fashions and open supply fashions equivalent to Mixtral to judge their effectiveness. GenAI fashions had been examined in a simulated QA function, the place they had been tasked with answering particular questions equivalent to “Did the agent redirect the shopper to on-line banking?”. The aim of this train was to evaluate how nicely these fashions might mimic human understanding and decision-making when verifying compliance with established pointers. By evaluating the efficiency of each the inner GPT mannequin and the open supply fashions, the workforce aimed to search out the best answer for enhancing customer support by means of automated AI-driven high quality management.
Advantages of the Databricks Knowledge Intelligence Platform for GenAI
The DataSentics workforce evaluated a number of choices for this answer, and in the end selected to deploy the Databricks Knowledge Intelligence Platform and Mosaic AI instruments at Česká spořitelna for a number of causes:
- Knowledge Administration and Governance Advantages: Unity Catalog makes information simply accessible for various fashions whereas holding delicate information underneath restricted entry.
- Complete Knowledge Processing Capabilities: the Databricks Platform helps your complete workflow of preprocessing of name heart information, from transcription to high quality management. This allows us to supply intermediate outcomes that may be leveraged for different fashions and tasks, equivalent to advertising and marketing, danger evaluation, regulatory compliance, and fraud detection.
- Mannequin Coaching and Assist: Databricks gives strong assist and experience for GenAI, together with mannequin structure and coaching capabilities. This made it an excellent platform for testing and deploying open supply fashions shortly, enabling us to experiment and iterate effectively.
- Ease of Cluster Creation: With Databricks, it’s simple to create clusters and deploy open-source fashions. This streamlines the experimentation course of and permits us to focus extra on mannequin efficiency and fewer on infrastructure administration.
Insights and Outcomes
All through the venture, we experimented with varied segmentation methods and gathered a number of beneficial insights:
- High quality of Enter Knowledge is Essential: The standard of the audio recordings diverse from shopper to shopper, with some talking quietly or from a distance, which might later have an effect on the accuracy of the transcription. Whisper or comparable techniques can assist remedy the issue.
- Class Definition is a Should: We realized that if classes can’t be simply outlined for people, it’s equally difficult for LLMs to grasp them. This strengthened the necessity for clear and exact class definitions to coach the fashions successfully.
- Open-Supply Fashions Ship Outcomes: Open-source fashions demonstrated that they may compete successfully with proprietary fashions like ChatGPT. This discovering is critical for companies seeking to optimize prices whereas nonetheless reaching high-quality outcomes.
What’s Subsequent
With GenAI instruments powered by Databricks Mosaic AI, Česká spořitelna staff are actually capable of acquire entry to solutions present in a spread of paperwork by way of “sensible search” performance. For instance, the buying workforce could have to seek the advice of tons of of pages of course of documentation on easy methods to management and approve funds to totally different international locations. Earlier than leveraging Databricks, it might take staff hours to search out the proper data they want. Now, RAG-powered search provides staff solutions inside seconds, together with citations and hyperlinks to the supply doc.
Trying forward, there are many alternatives to discover extra GenAI workloads at Česká spořitelna. We intention to create a strong integration between Databricks and Česká spořitelna’s inner database name heart recordings. It will unlock new use circumstances equivalent to churn detection, sentiment evaluation, and gross sales sign detection since Databricks is the go-to platform for streaming information. These day by day studies will enable Česká spořitelna to react to adjustments in actual time whereas reaching price reductions with improved high quality assurance of their name facilities.
This weblog put up was collectively authored by Petra Starmanova (Česká spořitelna), Tereza Mokrenova (DataSentics), Dalibor Karásek (DataSentics) and Joannis Paul Schweres (Databricks).