Generative AI has dominated headlines throughout 2024 and is set to remain in the spotlight well into 2025. While much of the focus has been on using Generative AI (GenAI) to answer questions, edit images, or summarize content, Ashling Partners has been exploring how Generative AI for Intelligent Automation can streamline business processes. We aim to automate and add intelligence to some or all processes to reduce the manual effort in completing a business process – providing immense value back to the business.
Although the applicability of Gen AI is extensive, the main questions that concern potential implementors revolve around the risk of an uncaptured error in a deterministic process and the ability to remediate those errors. Additionally, the extent of an error, if interpreted as a small mistake or something egregiously wrong and outside of the context of what was asked – so-called hallucinations – adds to hesitation in approaching Gen AI.
Many organizations hesitate due to these concerns. At Ashling Partners, our experts consistently assess Generative AI’s viability while identifying both its benefits and its drawbacks when paired with Intelligent Automation. Here’s what we’ve discovered so far.
Before you implement any use case, conduct a detailed statistical analysis of potential risks to understand possible outcomes and identify the necessary controls. Decide whether the use case needs human review— where value is provided by completing most of the work—or if programmatic controls is possible. Every use case is unique, so tailor your method of measuring both the efficacy and risk of the AI solution.
Comparing an AI algorithm’s performance to a human’s performance can also yield valuable insights. For instance, if Gen AI’s false negativity rate stands at 3%—the highest-risk quadrant—but a human’s rate for the same task is 5%, automation not only saves time but also reduces erroneous outputs.
At Ashling Partners, we view Gen AI as a powerful tool within an intelligent automation strategy. You can center entire processes on GenAI or use it to boost the automation rate in existing workflows by replacing rules-based steps prone to high error margins.
These tasks often require human review, which lowers the overall automation rate. Replacing them with higher-performing GenAI engines can improve each step’s performance by 50–60%, significantly increasing the value delivered.
One promising use case identified by our team is the use of Gen AI to extract, transform, and load (ETL) tasks of highly unstructured data. We have had incredibly positive results transforming raw data extracted from PDFs, excel, and web formats and standardizing and condensing the information to the desired format of the business. This is a common ask in automation use cases, where employees spend large amounts of time in non-value-added tasks of pulling information from tables to produce a report or create a business artifact on a routine basis.
Previous automation of these processes where the incoming data is unstructured has yet to be possible with rules-based automation or ML tools on the market due to the highly variable nature of the transformed content. Still, barriers remain.
Like humans, context is needed in prompt engineering to achieve a desired output. The more complex the ETL step, for example, if we are trying to standardize 100 columns rather than just 5-10 columns, the degradation in output quality is very clear. This can be mitigated slightly by adding the necessary context for the column matching, but the business value is considerably diminished overall. The complexity of the use case will determine how much use intervention will be required and the percentage of use cases that will require extensive review.
Another limitation affecting the complexity dimension is the size of the requests that a given Gen AI model permits. Models that can accept larger data sets and more extensive context cost considerably more. The infrastructure required to host these more capable models is also much greater.
Finally, where math is required, there is consensus in the community that Gen AI should not be used, as it does not perform math calculations and only considers the most statistically likely answer, which may or may not be the same.
Our team at Ashling Partners continues to assess the strengths and limitations of models in the market, such as Open AI’s GPT and Meta’s LLAMA2 and their different model tiers. Price points, performance, infrastructure requirements, business risks, and testing methodologies are some of the many considerations required when considering bringing Gen AI to your business effectively. Let us help you design and implement the GenAI and intelligent automation path that is best for your enterprise.
Subscribe to our newsletter for the latest news, insights and resources on Agentic AI and Intelligent Automation.