Azure Document Intelligence for Finance

You are logged in as: Free Trial User

This is a Members-Only Content

Azure AI Document Intelligence
for Finance

Author: Nicolas Boucher, April 2024

Introduction

Finance teams often deal with a large volume of documents (invoices, contracts, purchase orders, sales orders, credit notes…, etc.).

Unfortunately, in many companies, these documents are processed manually and require significant human labor.

In addition, it could create misplaced invoices, human errors in data entry, and delayed approvals, which add up to lost man hours and inefficiencies.

This article is an overview of a tool, Microsoft Azure AI Document Intelligence (formerly known as Form Recognizer), that can reduce manual document processing by automating the extraction and analysis of financial documents.

Document Intelligence streamlines operations, reduces errors, and accelerates decision-making.

Table of Contents

Technical Overview

Azure AI Document Intelligence is a cloud-based task-specific Azure AI service that uses machine-learning models to automate the processing of your data in applications and workflows.

Azure AI Document Intelligence leverages the power of machine learning, deep learning, Optical Character Recognition (OCR) and Supervised learning for document processing.

It has a wide range of applications for process automation, knowledge mining, and other industry-specific applications.

Core Functionalities

Workflow Overview

The following section provides a stepwise breakdown of the workflow, outlining the role of each component and the technical aspects of data extraction with Azure AI Document Intelligence.

Collecting Documents (unstructured data):

The starting point involves various unstructured documents in file formats supported by Azure Blob Storage, typically including PDF, JPG, PNG, Handwritten notes, and TIFF.

Data Storage in Azure Blob:

Azure Blob Storage is a cloud-based repository for these unstructured documents. It offers scalable and cost-effective storage for large volumes of data. The URLs of the uploaded documents can be fed to cognitive services for the next step.

Azure Function to Invoke Document Intelligence Service:

An Azure function acts as a serverless compute unit triggered by events. This workflow’s trigger is designed to respond to new document uploads within Azure Blob Storage.

The triggered Azure function utilizes the Azure Functions SDK to interact with other Azure services. The Document Intelligence service uses its REST API or a client library (available in various programming languages) to initiate document processing.

Document Intelligence:

Upon receiving the request from the Azure function, Azure AI Document Intelligence performs document layout analysis, analyzes the data type,

Layout Analysis utilizes pre-built or custom models to analyze the document structure and identify key features. This analysis might involve recognizing tables, key-value pairs, or specific data points relevant to the document type.
Data Type Analysis: It analyzes whether the data provided is a PDF, handwritten notes, image, or any other format.
Data Extraction: This process leverages OCR technology to convert the uploaded image file into machine-readable text and store the data in a database. The data could include key-value pairs like “Invoice Total” and their corresponding values, table data, or specific text blocks.

Structured Data for Onwards Processing/Analysis:

The Azure function receives the response from Azure AI Document Intelligence, which contains the extracted data in a structured format like JSON. This structured data is readily usable for further processing and analysis based on your needs, i.e., database storage, system integration, reporting, or logic app to initiate further processing (Payment in case of AP).

The diagram below presents the workflow of Azure Document Intelligence, which automates document processing and converts unstructured data into valuable insights for your applications.

See Azure AI Document Intelligence in action:

Document Analysis Models

Now that we know how it works, it is important to understand the different modalities of Azure AI Document Intelligence.

Document intelligence needs a context to extract the correct data from a document.

The type of model provides this context.

Fundamentally, there are two document analysis models: the prebuilt and the custom models.

A brief overview of both is below:

1. Prebuilt Models:

Prebuilt models are ready-to-use models for common document processing tasks.

These models are trained on a diverse dataset and can recognize common document types such as invoices, receipts, business cards, and forms.

They come pre-trained to extract critical fields and information from these documents accurately.

Fast to deploy: No need for training; you can use them right away.
Easy to use: Upload your documents and let the pre-trained model handle the extraction.
Well-suited for common scenarios: Ideal for invoices, receipts, and other predefined document types.

The most relevant prebuilt models for finance are “Invoices”, “Receipts” and “Contracts”.

2. Custom Models:

Custom models allow users to train their models tailored to their specific document processing needs.

Users can upload their dataset of documents and annotate them with the relevant fields they want to extract.

The custom model then learns from this annotated data to accurately recognize and extract the specified fields.

This is how the model works:

This offers greater control and accuracy for:

Uncommon document types: Train models to handle documents not covered by prebuilt options.
Complex layouts: Customize extraction for documents with non-standard layouts or workflows.
Improved accuracy: Train the model on your specific data to achieve higher accuracy for your unique documents.

Finance-based Utilisation

The possibilities with Microsoft Azure spread across all finance domains, from understanding tax compliances for different regions or industries to reducing manual processing.

Here are the most commonly used cases:

Automate ERP Data Entry

The most common use case of Azure document intelligence is reducing manual data entry through its OCR model.

Set up document intelligence to either a prebuilt or custom model, depending on how the documents to be processed are formed.
Get the documents analyzed by the Azure Document Intelligence engine.
Extract relevant information, including headers, tables, and line items.
Format the data and feed it to any database through API or flat tables
Then, using data entry tools like data loader or import option, you can quickly enter the data into an ERP system for further processing.

See the full documentation on the invoice model: here

Intelligent Readability of the Financial Statements

Extract key-value pairs from financial statements using General Layout API for consolidations, reporting, or Audit.

Set up a data enrichment pipeline within cognitive search that deals with financial statements.
Enrich the financial statements using general document skills.
Using the output to create a complex type field generates key-value pairs, tables, and entities (e.g., Sales, Net profit, P&L, balance sheet, etc.).
Feed the extracted information to a database.
Using a financial analysis tool or Python coding, you can perform analysis on it.
Additionally, you can follow the same steps to create intelligent search indexing of the financial statements that can be used for audit purposes.

Implementation of RPA (robotic payment automation)

We can set up logical sequences in Azure to create a robotic payment automation routine. It can be done in the following steps:

Form recognizer would extract the payable data and forward it for payment.
Set up a logic app to store invoices in Azure Blob.
Using Document Intelligence, extract the relevant data from the invoices
Format and feed the data to Azure Cosmos DB or any other DB tool using appropriate third-party integrations.
We can set up conditional workflows to execute the payment based on the stored data.

Vendor Bid Comparison

Through intelligent search indexing, essential information from vendor bids or supplier contracts can be extracted to analyze and compare different bids.

Set up a data enrichment pipeline within cognitive search that deals with vendor bids and contracts.
Enrich the document using general document API
Define and extract key-value pairs, tables, entities, and checklists from the documents
Format the extracted information as a structured database.
Using a generative AI tool or an analytical tool, compare different dimensions of the bids.
Using generative AI, create a recommendation and a summary report for the bidding process.

If you want to continue to learn more, here is a great webinar from Microsoft:

Azure Document Intelligence vs. Alternatives

Here’s a comparison table highlighting key features of Azure Document Intelligence and its prominent alternatives:

Feature	Deployment	Document Formats	Extraction Capabilities	Community & Support	Customization	Advanced Features	Accuracy	Pricing	Ease of Use	Security & Compliance
Azure AI Document Intelligence	Cloud-based	PDF, JPG, PNG, TIFF, BMP, Handwritten etc.	Key-value pairs, tables, text, custom models	Large community, good documentation	Yes, build custom models	Layout analysis, confidence scores	High, depends on document complexity	Free tier, pay-per-use	Easy to use with SDKs and libraries	Compliant with various industry standards
Amazon Comprehend	Cloud-based	PDF, JPG, PNG, etc.	Key-value pairs, entities, sentiment analysis	Large community, good documentation	Yes, custom entity recognizers	Redaction, multi-language support	High, varies based on document type	Free tier, pay-per-use	Easy to use with SDKs	Compliant with various standards
Google Cloud Natural Language API	Cloud-based	Various file formats	Text extraction, entity recognition	Large community, good documentation	Limited	Syntax analysis, topic modelling	Moderate	Pay-per-use	Easy to use with client libraries	Compliant with various standards
IBM Datacap	Cloud-based	Various	Forms, data extraction, workflow automation	Large community, extensive support	Yes, extensive customization	Batch processing, document routing	High	Subscription-based	More complex setup	Supports industry compliance
ABBYY Vantage	Cloud-based	Various	Invoices, documents, data extraction	Large community, good support	Yes, custom classifiers	Document classification	High	Subscription-based	More complex setup	Supports industry compliance
Tesseract (Open-Source in Python)	Open-source	Images (JPG, PNG, etc.)	Text extraction (OCR)	Active community, moderate support	Yes through Python	N/A	Lower accuracy compared to commercial options	Free	Requires programming knowledge	N/A
OpenCV (Open-Source in Python)	Open-source	Images (various)	Text extraction (OCR)	Active community, moderate support	Yes through Python	N/A	Can be customized through Python code	Free	Requires programming knowledge	N/A

Setting Up Azure AI Document Intelligence

Step-by-step instructions on how to implement Azure AI Document Intelligence for finance document processing

1. Sign up on Azure using the following link: https://signup.azure.com/

You can try it for free for 30 days but you do need to provide your credit card.

There is no installation needed to start using it as you can access Azure through your browser.

Video explaining the sign-up:

2. Setting up Azure Form Recognizer service in Azure Portal

From three bar on top left, click “create resource”

Search in the search bar “Document Intelligence”

Select “Document Intelligence (form recognizer)”
Create resource group

3. Pricing Tier & Connection Details

Select pricing tier, click create and get connection details.

4. Setting up extraction models using Document Intelligence studio

Open the link: https://documentintelligence.ai.azure.com/studio?WT.mc_id=aiml-8567-ayyonet
Select the model you want to create
Put in the descriptive information in the form
Upload documents.

Here is a short explanatory video:

Integrations

Azure Document Intelligence offers integrations with various services and tools to streamline your document processing workflow.

Azure Integrations

Document intelligence can be easily integrated with other Azure services for further processing. Some of the commonly used Azure services with document intelligence are

Azure Blob Storage for secure storage of documents
Azure Cognitive Services: Integrate with other Azure Cognitive Services like Computer Vision for advanced image analysis.
Power Automate will automate workflows based on events by document processing.

ERPS and CRMs

Third-party applications can be integrated with ERPs and CRMs using several integration methods, including APIs, SDKs, or custom integrations using platforms like Celigo.

Pricing and Cost Analysis

Azure Document Intelligence offers a flexible pricing model. It has three different pricing structures.

Pay as You Go: Pay only for what you use.

Commitment Tiers: Pay an upfront monthly fee for high-volume usage at a discount.

Commitment Tiers: Disconnected container

To learn more about the pricing you can also visit the following link:

https://azure.microsoft.com/en-us/pricing/details/ai-document-intelligence/

Security and Compliance

Considering the nature of financial data, ensuring security and compliance with best practices is important.

Azure Document Intelligence prioritizes encrypting data in transit and at rest, adhering to strict access controls, and following industry standards like SOC 2, HIPAA, and GDPR.

This ensures that your sensitive financial information, like invoice amounts and account numbers, remains protected throughout the processing pipeline.

Additionally, you have control over where your data is stored, allowing you to choose a region that aligns with your financial regulatory requirements.

For all features, the input data and results are deleted within 24 hours and not used for any other purpose.

For customer-trained models, customers can delete their models and associated metadata at any time by using the API.

Limitations and Challenges of Azure Document Intelligence in Finance

While Azure Document Intelligence offers a powerful solution for automating data extraction from financial documents, it’s essential to consider its limitations and challenges:

Accuracy:

Complex Layouts: Documents with non-standard layouts, tables, or handwritten text can pose challenges for accurate data extraction. The model might need help identifying and categorizing information correctly.
Data Ambiguity: Financial documents can contain abbreviations, jargon, or inconsistent formatting. This can lead to misinterpretations by the model, requiring human intervention for clarification.

Model Training and Management:

Custom Model Training: Creating and training custom models for specific document types requires time, effort, and a good-quality dataset. This can be a barrier for smaller finance teams.
Ongoing Maintenance: As your financial documents or formats evolve, prebuilt or custom models might need retraining to maintain optimal accuracy.

Security and Compliance:

Data Sensitivity: Financial documents often contain sensitive information. While Azure Document Intelligence offers robust security measures, ensuring compliance with specific financial regulations (e.g., PCI DSS) might require additional security protocols within the implementation. I recommend you to check your security policy to check for the compatibility of Azure security set up with your company’s requirements.

Integration Considerations:

ERP Integration: While Document Intelligence excels at data extraction, integrating it with your specific ERP system might require additional development or middleware tools. Ensuring seamless data exchange and workflow automation requires careful planning and implementation.

Cost Factors:

Pay-as-you-go Model: The service follows a pay-per-use pricing model. Processing large volumes of documents can lead to significant costs.

Conclusion

In conclusion, Azure Document Intelligence emerges as a powerful tool for document processing and analysis within the finance sector.

Leveraging advanced machine learning algorithms and cognitive services, Azure AI Document Intelligence can extract structured data from unstructured documents such as invoices, receipts, and forms.

Organizations can significantly improve efficiency, accuracy, and decision-making by automating tedious document processing tasks.

The greatest advantage is to benefit from the integration of Azure AI Document Intelligence with existing workflows and systems, especially if you are using the Microsoft environment, facilitating streamlined operations and enhanced productivity.