Script-based solutions like Exalate provide the flexibility to set up integrations for complex scenarios and intricate workflows.
However, with this flexibility, there’s also a need to maintain precision.
Building scripts from scratch, or sifting through documentation to look for the correct scripts isn’t always the most efficient approach. It can be time-consuming, especially when things don’t go as planned.
To address these issues, Exalate introduced AI Assist in October 2024 as a means to improve the script-based integration experience.
AI Assist and the Shift in Integration Success
AI Assist is embedded directly into Exalate’s Script mode interface to automate the script generation process and make it more efficient. It analyzes user input based on prompts and suggests scripts based on existing configurations and Exalate’s scripting API.
Since its launch, the goal was simple: to give users the power of scripting without the steep learning curve.
As more users began adopting AI Assist, we kept a close eye on feedback and performance. The more scripts it generated, the better we understood what worked and what didn’t.
What began as an integration enabler quickly evolved into a tool that learns and improves with every interaction.
As more and more users began trusting AI Assist to get their scripts right on the first try, acceptance rates climbed, confirming that AI Assist was making meaningful contributions.
Key Factors Behind AI Assist’s Success
Diving into user feedback revealed two major updates that made a noticeable difference.
- The Snippet Database: A collection of predefined code snippets that AI Assist uses to generate sync scripts. This database is carefully curated and constantly refined, ensuring scripts are accurate and up-to-date.
- Retrieval Algorithm: The rework of the retrieval algorithm reduced the time to generate code (or sync scripts) by nearly 50%. Now, users spend less time waiting and more time getting things done.
But we learned more than just how to improve the tool. Some users, for example, asked AI Assist questions meant for Aida, Exalate’s AI-powered documentation assistant.
This unexpected behavior gave us insight into how users interact with Exalate’s AI tools and where we could bridge the gap.
Tracking user actions like “Insert” or “Discard” allowed us to fine-tune AI Assist further, ensuring it aligned with user needs and expectations.
The Importance of Acceptance and Accuracy Rate in Integration Tech
When it comes to evaluating the success of AI Assist, two key metrics stand out: acceptance rate and accuracy rate.
- Acceptance Rate: Measures how often users accept AI-generated scripts. In short, did AI Assist help users achieve what they set out to do?
- Accuracy Rate: Measures how closely the generated script aligns with the user’s prompt. A script can be technically correct, but it is still declined if the prompt isn’t clear enough.
For example, if AI Assist produces a correct script but the user’s prompt lacks clarity, the acceptance rate might drop. However, the accuracy rate might still be high because the script itself was accurate.
Tracking these metrics revealed a crucial insight: about 30% of initially rejected answers were actually correct.
Each iteration of AI Assist used these insights to refine its output and improve performance.
Behind the AI: How we Build, Test, and Improve AI Assist
Every version of AI Assist addressed a new challenge, steadily enhancing the experience for users. Here’s a quick look at its evolution:
Version 1.4.0
Overhauled the retrieval system, cutting script generation time in half. Faster scripts meant users could move quicker, spending less time troubleshooting.
Version 1.5.0
- Improved AI Assist’s ability to understand the context of the entire user conversation instead of isolated prompts.
- Introduced better safeguard against malicious prompting.
Version 1.6.0
New reranking functionality: Added reranking functionality, which prioritizes the most relevant snippets from the snippet database, reducing the need for users to sift through suggestions.
Version 1.7.0 (Coming March 2025)
AI Assist will become more proactive, asking for clarification when prompts are unclear rather than generating scripts directly. This ensures users get scripts that align more closely with their needs.
Each step was a learning experience and brought us closer to a tool that not only generates scripts but understands user context.
The AI Assist Impact on Script-based Integrations and What’s Next
From the moment we introduced AI Assist, our goal was clear: make script-based integrations smoother, faster, and more accessible.
Early user feedback shows we’re on the right track. AI Assist is helping even non-technical users who aren’t familiar with scripting to implement integrations successfully.
So, what’s next?
The future of AI Assist is about more than just scripts. We’re building the AI Orchestrator: a unified interface that brings together all our AI tools into one powerful interface. Imagine having AI Assist and Aida in one place, ready to handle scripting questions, troubleshoot errors, or guide you through integrations without switching between assistants.
And that’s not all. We’re always exploring the latest advancements from OpenAI, Anthropic, and others to ensure AI Assist stays ahead of the curve.
Data and Privacy: How We Process and Use Data
Leveraging the power of AI always comes with the responsibility to protect and respect user privacy.
That’s why AI Assist is built with privacy at its core. It never accesses user data directly. The only information AI Assist uses is what you provide in your prompts or configuration.
To ensure compliance with industry standards, we:
- Track overall usage (e.g., prompt count, platform) without collecting personal data.
- Follow ISO 27001 rules for data storage.
- Anonymize logs, making them inaccessible to anyone who doesn’t need access.
Final Thoughts: The Journey Continues
Since its launch, AI Assist has transformed script-based integrations, making them smoother and more accessible for everyone.
With the AI Orchestrator and continuous improvements in the works, we can’t wait to see how AI Assist continues to evolve.
One thing is certain: we’ll keep listening, learning, and building so that Exalate users always have the best tools at their fingertips.
Read more about AI-powered integration here.
