The 7 November webinar, “Leveraging Artificial Intelligence to Enhance Your Audit Methodology,” hosted by the New York chapter of the IIA, featured speakers Diego Lascano, internal audit manager at Uber; Yelena Talmazan, managing director at K2 Integrity; and Christopher Ward, associate managing director at K2 Integrity. The event focused on the integration of AI into internal audit methodologies, highlighting its potential to streamline resources, improve productivity, and enhance risk management. The speakers provided insightful examples of AI tools and their applications in internal audit processes, underscoring the transformative impact of AI on the internal audit field.
The speakers discussed the current uses of AI:
- Generative AI, where the tools assist with individual productivity such as maximizing effort, working faster, improving quality, and reducing costs;
- Applied AI, where the tools assist with the team productivity such as AI store, custom AI, task automation, and improved outcomes; and,
- AI automation, where the tools have an organizational impact, including in customer service solutions, knowledge management, marketing automation, AI applications, and retrieval-augmented generation (RAG).
They also highlighted future uses of AI:
- AI innovation, where the tools have a customized impact, including new AI features in products, enhanced product value, customer activations, assisted fine-tuning, and context connectors; and,
- Custom AI, involving proprietary AI tools such as custom model training, embedding models into features/products, agents (automation in operations), and high-level monitoring and oversight.
The discussion focused on the benefits and drawbacks of in-house (e.g., infra+, TP/own AI) vs. third-party AI tools (Gemini, llama, ChatGPT). The in-house tools offer customization specific to needs and full control over development and data but require a higher initial investment of time and resources. Third-party tools provide organizations benefits such as access to expertise, a reduced need for internal technical resources, and faster implementation, but offer less customization and increased privacy risks. All AI tools have challenges, including hallucinations, bias, overfitting, and data poisoning.
As part of the discussion, the speakers focused on how different organizations are using AI tools, including the stages of AI program implementation, the AI tools in use, AI methodology, the positive impacts of AI, and the benefits of having an AI roadmap.
The moderator polled the attendees about their organizations’ use of AI. Here are some of the results:
Stage of AI Program
- Many companies are in the early stage of their AI program
- Some are starting to use AI tools for research and administrative work
- A small number of respondents use AI tools proactively
- A few companies prohibit the use of AI
AI Tools Used
- Common AI tools mentioned include:
- ChatGPT
- CoPilot
- Internal company tools
- Other tools like Llama, Gemini, and Claude
AI Methodology
- Companies are at different stages with their AI methodology, including:
- Early stages of drafting
- Piloting basic AI projects
- Basic AI methodology already exists
- Established AI methodology with robust use
- A few said that they are piloting complex AI projects
Positive Impacts of AI
- Those using AI found it helpful in:
- Speeding up daily tasks
- Research
- Monitoring
- Automating more complex tasks
AI Roadmap
- Over half of the organizations have an AI roadmap in various stages:
- Awareness and exploration stage
- Assessment and strategy development
- Pilot projects and proof of concept
- Deployment and integration
- Optimization and scaling
- Governance and risk management
- Continuous improvement and innovation
The speakers highlighted several key challenges faced by organizations in leveraging AI tools. These same challenges were also highlighted in the attendees’ responses:
- Lack of AI Adoption: About 20% of respondents indicated that they do not use AI tools at all. This gap in AI adoption could be due to various factors such as lack of awareness, resources, or perceived value.
- Early Stages of AI Programs: Many companies are still in the early stage of their AI programs. This stage—which involves drafting AI methodologies, piloting basic AI projects, and starting to use AI tools for research and administrative work—presents challenges in terms of developing robust AI strategies and integrating AI into daily operations.
- Prohibition of AI Use: Policies that prohibit the use of AI can be a significant barrier to leveraging AI’s potential benefits and may stem from concerns about data privacy or security or from ethical considerations.
- Uncertainty and Lack of Knowledge: A considerable number of respondents indicated that they do not know whether their company has an AI methodology or program. This uncertainty can hinder the effective implementation and utilization of AI tools.
- Diverse AI Methodologies: Companies are at different stages regarding their AI methodologies, ranging from early drafting stages to established and robust use. This diversity indicates that while some organizations are making significant progress, others are still thinking about developing or struggling to develop and implement effective AI strategies.
- Challenges in AI Roadmap Development: While more than 50% of organizations have AI roadmaps in various stages of development, others do not have a clear plan for the future. This lack of focus can lead to challenges in aligning AI initiatives with business goals and ensuring a structured approach to AI adoption.
There are several strategies to overcome the challenges faced by organizations in leveraging AI tools:
- Develop a Clear AI Strategy: For companies in the early stages of their AI programs, it is crucial to develop a clear AI strategy that outlines the goals, objectives, and implementation plans. This strategy should include a roadmap for integrating AI into various business processes and a timeline for achieving key milestones.
- Increase AI Awareness and Training: To address the lack of AI adoption, organizations should invest in training programs and workshops to raise awareness about the benefits and applications of AI tools. This can help employees understand how AI can enhance their work and encourage them to adopt these tools.
- Encourage Experimentation and Pilots: Organizations should encourage experimentation with AI tools by running pilot projects and proof-of-concept initiatives. This allows companies to test the effectiveness of AI tools in different scenarios and gather valuable insights before scaling up their AI initiatives.
- Establish AI Governance and Risk Management: To address concerns about data privacy, security, and ethical considerations, organizations should establish robust AI governance frameworks. This includes setting up policies and procedures for AI use, monitoring AI applications for compliance, managing risks associated with AI deployment, and performing internal audits.
- Foster Collaboration Across Departments: Successful AI adoption requires collaboration between various departments, including IT, operations, marketing, customer service, and internal audit. By fostering cross-functional collaboration, organizations can ensure that AI initiatives are aligned with business goals and that different teams can contribute their expertise to AI projects.
- Invest in AI Infrastructure and Tools: Organizations should invest in the necessary infrastructure and tools to support AI initiatives. This includes acquiring advanced AI tools, setting up data management systems, and ensuring that the IT infrastructure can handle the computational demands of AI applications.
- Monitor and Evaluate AI Impact: To enable AI initiatives to deliver the desired outcomes, organizations should continuously monitor and evaluate the impact of AI tools—tracking key performance indicators, gathering feedback from users, and making adjustments to AI strategies based on the insights gained.
By implementing these strategies, organizations can overcome the challenges associated with AI adoption and fully leverage the potential of AI tools to enhance their operations and achieve their business objectives.
