<h1>Save and restore</h1>

&nbsp;

The&nbsp;<code>mindee.Client</code>&nbsp;includes methods for saving and restoring api calls. Those methods are helpful if you want to perform tests or if you simply want to reload Invoices objects corresponding to your files without performing any additional request.

&nbsp;

&nbsp;

<h2>Save a call</h2>

&nbsp;

The <code>mindee.Response</code> object returned by the <code>mindee.Client</code> when parsing a file is responsible for wrapping all the data from Mindee API calls. It&nbsp;includes a <code>dump</code> method that can be used as follows:

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(invoice_token="your_invoices_api_token_here")

invoice_data = mindee_client.parse_invoice(open("./sample.pdf", 'rb'), input_type="file")

invoice_data.dump("./test_dump.json")</code></pre>

&nbsp;

&nbsp;

<h2>Restore a call</h2>

&nbsp;

To restore a client response object, you can use the <code>mindee.Response</code> as follows:

&nbsp;

<pre>
<code class="language-python">from mindee import Response

invoice_data = Response.load("./test_dump.json")</code></pre>

&nbsp;

&nbsp;

Save and restore Mindee Invoice API response with Mindee python helper library

Save and restore

<h1>Extract data from invoices with Mindee&#39;s API using Python&nbsp;</h1>

&nbsp;

In this tutorial, you will learn how to extract data from invoices (pdfs or images) using python. This can help you automatically process invoices in any accounting software.

&nbsp;

If you want to get more details about how to use the mindee python library:

<ul>
	<li>1.&nbsp;<a href="https://mindee.com/documentation/apis/invoice-parsing/python-invoice/getting-started">Python SDK overview</a></li>
	<li>2.&nbsp;<a href="https://mindee.com/documentation/apis/invoice-parsing/python-invoice/multi-pages-invoice-pdfs">Handle multi pages invoices</a></li>
	<li>3.&nbsp;<a href="https://mindee.com/documentation/apis/invoice-parsing/python-invoice/invoice-fields">Extracted invoice fields</a></li>
	<li>4.&nbsp;<a href="https://mindee.com/documentation/apis/invoice-parsing/python-invoice/save-and-restore">Save and restore Invoice objects</a></li>
	<li>5.&nbsp;<a href="https://mindee.com/documentation/apis/invoice-parsing/python-invoice/data-validation">Invoice data validations</a></li>
	<li>6.&nbsp;<a href="https://mindee.com/documentation/apis/invoice-parsing/python-invoice/performances-benchmark">Benchmark Invoice OCR performances</a></li>
</ul>

&nbsp;

&nbsp;

<h2>Tutorial</h2>

&nbsp;

The final script can be used within any python middleware application or REST API serving your frontend interfaces.

&nbsp;

In the end of this tutorial, you will learn how to transform a pdf into an image file, and how to highlight features on it to make human validation easier.&nbsp;

&nbsp;

The final result looks like this:

&nbsp;

<img alt="extract data invoices python mindee api" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2020/10/29/ezgifcom-gif-maker.gif" />

&nbsp;

&nbsp;

<h2>API Prerequisites</h2>

&nbsp;

<ol>
	<li>You&rsquo;ll need a free Mindee account. <a href="https://platform.mindee.net/signup">Sign up</a> and confirm your email to login.</li>
	<li>An invoice, look into your mailbox, download one <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2020/10/29/1p_invoice_b.pdf" target="_blank">here</a> or <a href="https://www.google.com/search?q=sample+invoice">check out on google image</a></li>
</ol>

&nbsp;

&nbsp;

<h2>Setup the project</h2>

&nbsp;

Create an empty directory wherever you want in your laptop, we&rsquo;ll call ours &ldquo;python_invoice&rsquo;.

&nbsp; 
We recommend you to create a virtual environment before going to the next step, in order to avoid installing the required packages globally on your machine.To do so, run the following command:

&nbsp;

(macOS and Linux)

<pre>
<code class="language-bash">python -m venv venv</code></pre>

 
(windows)

<pre>
<code class="language-bash">py -m venv venv</code></pre>

&nbsp;

Then, you&#39;ll need to activate your venv, so that pip will install packages in that folder.

&nbsp;

(macOS and Linux)

<pre>
<code class="language-bash">source venv/bin/activate</code></pre>

 
(Windows)

<pre>
<code class="language-bash">.\env\Scripts\activate</code></pre>

&nbsp;

&nbsp;

We need now to Install mindee python library&nbsp;from PyPi using pip, a package manager for Python.

&nbsp;

<pre>
<code class="language-bash">pip install mindee</code></pre>

&nbsp;

If the install throws an error, you might need to upgrade your version of pip. To do so, run:

&nbsp;

<pre>
<code class="language-bash">pip install --upgrade pip </code></pre>

&nbsp;

Don&#39;t have pip installed? Try installing it, by running this from the command line:

&nbsp;

<pre>
<code class="language-bash">$ curl https://bootstrap.pypa.io/get-pip.py | python</code></pre>

&nbsp;

That&rsquo;s it for the setup, let&rsquo;s call the API.

&nbsp;

&nbsp;

<h2>Call the Mindee Invoice API</h2>

&nbsp;

&nbsp;

Login onto the [Mindee dashboardt](https://platform.mindee.net/) and enter the&nbsp;Invoice API environment by clicking the following card.

&nbsp;

&nbsp;

<img alt="mindee invoice api card" src="https://lh3.googleusercontent.com/JwH29ZumYioDxoUEJ_9BcyZjIMRHq9b-xGf5FWrNS1ZyJ1HKZ66Ihwpxiyop3MB6tNchMr9Py4VbQk2cKQ80nk4Gy1Q0nOe0zs0avA101kBA1uagx9JmTQ2A0FIpC62MxNhi1rKz" style="height:294px; width:300px" />

&nbsp;

&nbsp;

If you don&rsquo;t have an API token yet for this API, go to the &ldquo;credentials&rdquo; section and create a new API Key.&nbsp;

&nbsp;

&nbsp;

<img alt="new api token" src="https://lh3.googleusercontent.com/CWRZZFfY2uVd5Lyx-wxxR_EjWcw6ZBryFhhpNMwP1DBO5LQEEUBfqos2sPjCE4BQ6AdxelSzaybQEl5EGGXDL-eqwr5qXA3dtgn8_TquDVfVp8xFMsjV_npDbTblZ-wXTgOkwoEy" style="height:146px; width:300px" />

&nbsp;

&nbsp;

&nbsp;

Open the &ldquo;main.py&rdquo; file you created a few minutes ago, and paste your sample code.&nbsp; The path to your invoice will be placed in the variable &quot;filepath&quot; since we call it several times. Your main.py file&nbsp;should look&nbsp;like:

&nbsp;

&nbsp;

<pre>
<code class="language-python">from mindee import Client

filepath = "/path/to/your/file"
invoice_token = "my-token-here"

mindee_client = Client(invoice_token=invoice_token)


if __name__ == "__main__":
 
 response = mindee_client.parse_invoice(filepath)

 print(response.invoice)</code></pre>

&nbsp;

&nbsp;

Replace the &ldquo;path/to/my/file&rdquo; placeholder in the code with the path of the invoice pdf or image you want to parse. Replace the &ldquo;my-token-here&rdquo; with the API token you created previously in the platform.

&nbsp;

&nbsp;

You can get back to your console and run the script

&nbsp;

&nbsp;

<pre>
<code class="language-bash">python main.py</code></pre>

&nbsp;

&nbsp;

You should see the details of the invoice appearing in your console:

&nbsp;

<pre>
<code class="language-bash">-----Invoice data-----
Filename: 1p_invoice_b.pdf 
Invoice number: A-1414 
Total amount including taxes: 2608.2 
Total amount excluding taxes: 2415.0 
Invoice date: 2018-09-25
Supplier name: DESIGNS TURNPIKE CO
Taxes: 193.2 8.0%
Total taxes: 193.2
----------------------</code></pre>

&nbsp;

Now let&rsquo;s take a look at the Invoice object.

&nbsp;

<h2>&nbsp;</h2>

<h2>Extracted invoice data</h2>

&nbsp;

The API returns a list of different fields in the invoice (total amount, taxes, date &hellip;). You can find this list in the &ldquo;documentation&rdquo; section in Mindee platform.

&nbsp;

The Invoice object under the&nbsp;<code>Response.invoice</code>&nbsp;attribute has all the feature you need to get the invoice data and coordinates in the image to highlight the results.

&nbsp;

Here is a piece of code to show you how to get some of them:

&nbsp;

<pre>
<code class="language-python">from mindee import Client

filepath = "/path/to/your/file"
invoice_token = "my-token-here"

mindee_client = Client(invoice_token=invoice_token)


if __name__ == "__main__":
 
 response = mindee_client.parse_invoice(filepath)

 # To get the total amount including taxes value (float), ex: 14.24
 print(response.invoice.total_incl.value)
 
 # To get the total amount excluding taxes value (float), ex: 10.21
 print(response.invoice.total_excl.value)
 
 # To get the total tax amount value (float), ex: 8.42
 print(response.invoice.total_tax.value)
 
 # To get the list of taxes
 print(response.invoice.taxes)

 # Loop on each Tax field
 for tax in taxes:
 # To get the tax amount
 print(tax.value)
 
 # To get the tax code for from a tax object
 print(tax.code)
 
 # To get the tax rate
 print(tax.rate)
 
 # To get the invoice date of issuance (string)
 print(response.invoice.invoice_date.value)
 
 # To get the invoice due date (string)
 print(response.invoice.invoice_date.value)</code></pre>

&nbsp;

Run the script and check out the results in your console!

&nbsp;&nbsp;

Now we are going to add a piece of code that will highlight the features extracted on the pdf. It can help you or your users to very quickly validate the data extraction.

&nbsp;

&nbsp;

<h2>Convert pdf into an image</h2>

&nbsp;

If you&rsquo;re using a pdf, we need first to convert the pdf into an image, as it&rsquo;s more convenient to use for highlighting the features on it. If your invoice is an image, you can skip this part.

&nbsp;

To convert the pdf into a jpeg we are going to use the <a href="https://github.com/Belval/pdf2image">pdf2image</a>&nbsp;python library; First install it in your environment:

&nbsp;

<pre>
<code class="language-bash">pip install pdf2image</code></pre>

&nbsp;

Important note: Dealing with pdfs is not easy and this library might not work directly because it sometimes requires some extra dependencies. Check out the github to make sure you installed it correctly.

&nbsp;

Once installed, we are going to create an image out of our pdf:

&nbsp;

&nbsp;

<pre>
<code class="language-python">from pdf2image import convert_from_path, convert_from_bytes
images = convert_from_path(filepath)</code></pre>

&nbsp;

&nbsp;

The images object is an array of Pillow images. Pillow is an image library for python. As we want to use openCV for highlighting the features on the image, we need to convert the pillow image into an OpenCV image. OpenCV is based on the numpy library, and to convert it we are going to use numpy directly.

&nbsp;

If you don&rsquo;t have numpy installed on your environment:

&nbsp;

&nbsp;

<pre>
<code class="language-bash">pip install numpy</code></pre>

&nbsp;

&nbsp;

Now let&rsquo;s convert the Pillow image into an opencv format using numpy:

&nbsp;

&nbsp;

<pre>
<code class="language-ruby">import numpy as np
from pdf2image import convert_from_path
 
def pdf_to_cv_image(filepath, page=0):
 image = convert_from_path(filepath)[page]
 return np.asarray(image)</code></pre>

&nbsp;

 
 
&nbsp;

<h2>Highlight features on the image</h2>

&nbsp;

Let&rsquo;s try to highlight the features as if someone did it with a pen.

&nbsp;

First, you&rsquo;ll need to install the computer vision python library OpenCV if you don&rsquo;t have it already installed in your env. To do so, run:

&nbsp;

&nbsp;

<pre>
<code class="language-bash">pip install opencv-python</code></pre>

&nbsp;

Each field extracted in the <code>Response.invoice</code> object has a <code>Field.bbox</code> object. It&#39;s as simple list of bounding boxes relative vertices (% of the image width and height).

&nbsp;

Check out what&#39;s inside for the total_incl.bbox for example, doing:

&nbsp;

<pre>
<code class="language-python">print(invoice_data.invoice.total_incl.bbox)</code></pre>

&nbsp;

Now that we know how to access the bounding boxes coordinates, we are going to create our highlighter function in which we can feed any field bounding boxes&nbsp;in different steps, as follows:

&nbsp;

<ol>
	<li>Check if the file is a pdf or an image and load it as an openCV image in both scenarios</li>
	<li>Create a mask image</li>
	<li>Loop on each feature coordinates and draw the feature rectangle on our mask</li>
	<li>Overlay the mask and original image with alpha</li>
	<li>Display image to the user</li>
</ol>

&nbsp;

Note: each coordinate returned by the API is relative (in % of the image). You&#39;ll see there is a relative to absolute conversion in the code.

&nbsp;

Here is the code step by step:

&nbsp;

<pre>
<code class="language-ruby">def highlight_features(img_path, coordinates):
 # step 1: Check if the file is a pdf or an image and load it as an openCV image in both scenarios
 if img_path.endswith(".pdf"):
 cv_image = pdf_to_cv_image(img_path)
 else:
 cv_image = cv2.imread(img_path)

 # step 2: create mask image
 overlay = cv_image.copy()
 h, w = cv_image.shape[:2]

 # step 3: Loop on each feature coordinates and draw the feature rectangle on our mask
 for coord in coordinates:
 if len(coord):
 # Create points absolute coordinates
 pt1 = (int(w*coord[0][0]), int(h*coord[0][1]))
 pt2 = (int(w*coord[2][0]), int(h*coord[2][1]))
 cv2.rectangle(overlay, pt1, pt2, (70, 230, 244), cv2.FILLED)

 # step 4: Overlay the mask and original image with alpha
 final_image = cv2.addWeighted(overlay, 0.5, cv_image, 0.5, 0)

 # step 5: Display image to the user
 cv2.imshow("highlghted_image", cv2.resize(final_image, (600, int(600*h/w))))
 cv2.waitKey(0)</code></pre>

 
&nbsp;

Finally, we just have to modify a bit the code for executing the highlighting function and printing the results. Here is what the final code:

&nbsp;

&nbsp;

<pre>
<code class="language-python">from mindee import Client
import cv2
import numpy as np
from pdf2image import convert_from_path


def pdf_to_cv_image(filepath, page=0):
 image = convert_from_path(filepath)[page]
 return np.asarray(image)


def highlight_features(img_path, coordinates):
 # step 1: Check if the file is a pdf or an image and load it as an openCV image in both 
 scenarios
 if img_path.endswith(".pdf"):
 cv_image = pdf_to_cv_image(img_path)
 else:
 cv_image = cv2.imread(img_path)

 # step 2: create mask image
 overlay = cv_image.copy()
 h, w = cv_image.shape[:2]

 # step 3: Loop on each feature coordinates and draw the feature rectangle on our mask
 for coord in coordinates:
 if len(coord):
 pt1 = (int(w*coord[0][0]), int(h*coord[0][1]))
 pt2 = (int(w*coord[2][0]), int(h*coord[2][1]))
 cv2.rectangle(overlay, pt1, pt2, (70, 230, 244), cv2.FILLED)

 # step 4: Overlay the mask and original image with alpha
 final_image = cv2.addWeighted(overlay, 0.5, cv_image, 0.5, 0)

 # step 5: Display image to the user
 cv2.imshow("highlghted_image", cv2.resize(final_image, (600, int(600*h/w))))
 cv2.waitKey(0)


filepath = "/path/to/your/file"
invoice_token = "my-token-here"

mindee_client = Client(invoice_token=invoice_token)


if __name__ == "__main__":
 
 response = mindee_client.parse_invoice(filepath)

 print(response.invoice)

 highlight_features(
 filepath,
 [
 response.invoice.total_incl.bbox,
 response.invoice.total_excl.bbox,
 response.invoice.invoice_number.bbox,
 response.invoice.supplier.bbox,
 response.invoice.invoice_date.bbox,
 response.invoice.due_date.bbox
 ]
 )
</code></pre>

 
&nbsp;

And the final result!

&nbsp;

<img alt="" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2020/10/29/final_invoice_python.jpg" /> 
&nbsp;

<h2>Conclusion</h2>

&nbsp;

In just over a few seconds, an invoice was parsed and the features were highlighted.&nbsp; 
&nbsp;

If you wanted to use this kind of script to display results to your users, i&#39;d advise you to do the highlighting in the front-end application, as sending images back from your middleware is not the best option because of payload sizes. The other solution would be to store the final image using cv2.imwrite(...) but it would make your client download the result.&nbsp;

&nbsp;

If you have questions, please reach out to us in the chat widget in the bottom right.

Tutorial on how to extract data from invoices (pdfs or images) using python. This can help you automatically parse invoices in any web applications.

<h1>Getting started</h1>

<h2>&nbsp;</h2>

&nbsp;

Get started with invoice parsing in python.

&nbsp;

<h2>1. Install the mindee python helper library</h2>

&nbsp;

Install from PyPi using&nbsp;<a href="https://pip.pypa.io/en/latest/" rel="nofollow">pip</a>, a package manager for Python.

&nbsp;

<pre>
<code class="language-bash">pip install mindee</code></pre>

&nbsp;

Don&#39;t have pip installed? Try installing it, by running this from the command line:

&nbsp;

<pre>
<code class="language-bash">$ curl https://bootstrap.pypa.io/get-pip.py | python</code></pre>

&nbsp;

Getting started with the Mindee API couldn&#39;t be easier. Create a&nbsp;<code>Client</code>&nbsp;and you&#39;re ready to go.

&nbsp;

&nbsp;

&nbsp;

<h2>2. Instantiate your Client</h2>

&nbsp;

The&nbsp;<code>mindee.Client</code>&nbsp;needs your API credentials. You can either pass these directly to the constructor (see the code below) or via environment variables.

&nbsp;

Depending on what type of document you want to parse, you need to add specifics auth token for each endpoint.

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(
 invoice_token="your_invoices_api_token_here",
 raise_on_error=True
)</code></pre>

&nbsp;

<ul>
	<li>invoice_token:&nbsp;(string) API key for invoices endpoint</li>
	<li>raise_on_error: (boolean, default True) Specify wheter or not raising an Exception when HTTP errors occur</li>
</ul>

&nbsp;

&nbsp;

We suggest storing your credentials as environment variables. Why? You&#39;ll never have to worry about committing your credentials and accidentally posting them somewhere public.

&nbsp;

&nbsp;

&nbsp;

<h2>3. Parse an invoice</h2>

&nbsp;

Using the parse_invoice&nbsp;method from your client object, you can pass any image or pdf file (see <a href="https://mindee.com/documentation/apis-overview/input-data">inputs file</a>&nbsp;for pdf pages number limits) to get the invoice data.

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(
 invoice_token="your_invoices_api_token_here",
 raise_on_error=True
)

parsed_data = mindee_client.parse_invoice("/path/to/file")
</code></pre>

&nbsp;

&nbsp;

<h3>Input types</h3>

&nbsp;

You can pass your input file in three ways:

&nbsp;

From file path

<pre>
<code class="language-python">invoice_data = mindee_client.parse_invoice('/path/to/file', input_type="path")</code></pre>

&nbsp;

&nbsp;From a file object

<pre>
<code class="language-python">with open('/path/to/file', 'rb') as fp:
 invoice_data = mindee_client.parse_invoice(fp, input_type="file")
</code></pre>

&nbsp;

From a base64

<pre>
<code class="language-python">invoice_data = mindee_client.parse_invoice(base64_string, input_type="base64")</code></pre>

&nbsp;

&nbsp;

invoice_data structure

&nbsp;

The invoice_data object returned by the&nbsp;<code>parse_invoice</code>&nbsp;method contains the following elements:

&nbsp;

invoice_data.invoice

&nbsp;

The document attribute&nbsp;is the Invoice object constructed by gathering all the pages into a single document.

&nbsp;

<pre>
<code class="language-python">invoice_data.invoice # returns a unique object from class Invoice</code></pre>

<h4>&nbsp;</h4>

&nbsp;

invoice_data.invoices

&nbsp;

For multi pages pdf, the &#39;pages&#39; attribute is a list of documents objects, each object is constructed using a unique page of the pdf

&nbsp;

<pre>
<code class="language-python">parsed_data.invoices # [Invoice, Invoice ...] </code></pre>

<h4>&nbsp;</h4>

&nbsp;

invoice_data.http_response

&nbsp;

Contains the full Mindee API HTTP response object

&nbsp;

<pre>
<code class="language-python">parsed_data.http_response # full HTTP request object </code></pre>

&nbsp;

&nbsp;

&nbsp;

<h2>4. Display the results</h2>

&nbsp;

You only have to print your invoice object to display the different extracted fields:

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(
 invoice_token="your_invoices_api_token_here",
 raise_on_error=True
)

invoice_data = mindee_client.parse_invoice("/path/to/invoice")

print(invoice_data.invoice)</code></pre>

&nbsp;

&nbsp;

Get started with invoice parsing in python and Mindee Invoice API.

Getting started

<h1>Multi pages pdfs</h1>

&nbsp;

You might want to know how to handle multi pages pdfs for many reasons:

&nbsp;

<ol>
	<li>How to handle invoices pdfs with a high number of pages</li>
	<li>How to get a single invoice result for a multi pages pdf invoice</li>
	<li>How to get an invoice result for each page</li>
</ol>

&nbsp;

&nbsp;

<h2>How to handle invoices pdfs with a high number of pages</h2>

&nbsp;

For performances purposes, Mindee APIs don&#39;t allow pdf files as input if they contains too many pages. The <code>parse_invoice</code>&nbsp;method includes a&nbsp; helper for cutting pdf pages in order to perform requests whatever the number of pages in the pdf.

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(invoice_token="your_invoices_api_token_here")

invoice_data = mindee_client.parse_invoice("./sample.pdf", cut_pdf=True)</code></pre>

&nbsp;

&nbsp;

cut_pdf: (Bool, default True) If set to True and the input file is a pdf with more than 3 pages, the pdf will be automatically reconstructed as a 3 pages pdf including the first and the two last pags. If set to False, if the pdf input file&nbsp;contains too many pages, an Exception will be thrown.

&nbsp;

&nbsp;

<h2>How to get a single invoice result for a multi pages pdf invoice</h2>

&nbsp;

Each response from mindee parsing methods includes a reconstructed document from all the pages. To construct this object, the <code>mindee.Response</code> class will loop over each page&#39;s result&nbsp;and get for each field the maximum confidence score result. Finally it will construct the Invoice object and you can access it this way:

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(invoice_token="your_invoices_api_token_here")

invoice_data = mindee_client.parse_invoice("./3pages.pdf", cut_pdf=True)

single_invoice_for_multipages = invoice_data.invoice</code></pre>

&nbsp;

&nbsp;

&nbsp;

<h2>How to get an invoice result for each page</h2>

&nbsp;

If you know that your multi pages pdf file contains a single invoice for each page, you can still access a&nbsp;<code>mindee.Invoice</code> object for each pages by doing:

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(invoice_token="your_invoices_api_token_here")

invoice_data = mindee_client.parse_invoice("./3pages.pdf", cut_pdf=True)

for invoice in invoice_data.invoices:
 print(invoice)</code></pre>

&nbsp;

You might want to know how to handle multi pages invoice pdfs for many reasons, how to handle invoices pdfs with a high number of pages, how to get a single invoice result for a multi pages pdf invoice, how to get an invoice result for each page

Multi pages invoice pdfs

<h1>Invoice&nbsp;fields</h1>

&nbsp;

&nbsp;

<h2>General fields structure</h2>

&nbsp;

Each invoice object contains a set of different fields.

&nbsp;

To acces an invoice object, you need to create a <code>mindee.Client</code>&nbsp;and call the <code>Client.parse_invoice</code>&nbsp;method:

&nbsp;

<pre>
<code>from mindee import Client

# Instantiate your client
mindee_client = Client(invoice_token="your_token_here")

# Call the Mindee invoice endpoint and parse the result
invoice_data = mindee_client.parse_invoice("/path/to/my/file")</code></pre>

&nbsp;

Whatever the field type, each of them contains the four following attributes:

&nbsp;

<ul>
	<li>Value: (Str or Float depending on the field type), corresponds to the field value. Set to None if the field was not extracted.</li>
	<li>Probability: (Float), confidence score of the field prediction.</li>
	<li>bbox:(Array[Float]), contains the relative vertices coordinates of the bounding box containing the field in the image. If the field is not written, the bbox is an empty array.&nbsp;</li>
	<li>reconstructed: (Bool), True if the field was reconstructed using other fields.</li>
</ul>

&nbsp;

Depending on the Field type, there can be extra attributes.

&nbsp;

&nbsp;

&nbsp;

<h2>Total amounts</h2>

&nbsp;

&nbsp;Invoice.total_incl:&nbsp;Total amount including taxes

<pre>
<code class="language-python"># To get the total amount including taxes value (float), ex: 14.24
total_incl = invoice_data.invoice.total_incl.value
</code></pre>

&nbsp;

&nbsp;Invoice.total_excl:&nbsp;Total amount excluding taxes

<pre>
<code class="language-python"># To get the total amount excluding taxes value (float), ex: 10.21
total_excl = invoice_data.invoice.total_excl.value</code></pre>

&nbsp;

Invoice.total_tax: Total tax value reconstructed from tax lines

<pre>
<code class="language-python"># To get the total tax amount value (float), ex: 8.42
total_tax = invoice_data.invoice.total_tax.value</code></pre>

<img src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2020/10/26/image-20201026151427-1.gif" />

&nbsp;

&nbsp;

<h2>Taxes</h2>

&nbsp;

invoice.taxes:&nbsp;Array of Tax fields

Each of the tax field has two extra attributes:

&nbsp;

<ul>
	<li>code: (String), Optional tax code. (HST, GST... for canadian; City Tax, State tax for US etc..)</li>
	<li>rate: (Float), Optional tax rate.</li>
</ul>

&nbsp;

<pre>
<code class="language-python"># To get the list of taxes
taxes = invoice_data.invoice.taxes

# Loop on each Tax field
for tax in taxes:
 # To get the tax amount
 tax_amount = tax.value

 # To get the tax code for from a tax object
 tax_code = tax.code
 
 # To get the tax rate
 tax_rate = tax.rate</code></pre>

&nbsp;

&nbsp;

&nbsp;

<h2>Dates</h2>

&nbsp;

Each date field comes with an extra attribute:

&nbsp;

<ul>
	<li>date_object: (Datetime), datetime object from python datetime.date library</li>
</ul>

&nbsp;

&nbsp;

&nbsp;invoice.invoice_date: Date of issuance of the invoice

<pre>
<code class="language-python"># To get the invoice date of issuance (string)
invoice_date = invoice_data.invoice.invoice_date.value</code></pre>

&nbsp;

invoice.due_date: Payment due date of the invoice

<pre>
<code class="language-python"># To get the invoice due date (string)
due_date = invoice_data.invoice.invoice_date.value</code></pre>

&nbsp;

&nbsp;

&nbsp;

<h2>Supplier informations</h2>

&nbsp;

&nbsp;Invoice.supplier:&nbsp;Supplier name as written in the invoice (logo or supplier infos

<pre>
<code class="language-python"># To get the supplier name
supplier_name = invoice_data.invoice.supplier.value</code></pre>

&nbsp;

&nbsp;

Invoice.payment_details:&nbsp;List of invoice&#39;s supplier&nbsp;payment details informations.

Each object in the list contains extra attributes:

&nbsp;

<ul>
	<li>iban: (String)</li>
	<li>swift: (String)</li>
	<li>routing_number: (String)</li>
	<li>account_number: (String)</li>
</ul>

&nbsp;

<pre>
<code class="language-python"># To get the list of payment details
payment_details = invoice_data.invoice.payment_details

# Loop on each object
for payment_detail in payment_details:
 # To get the IBAN
 iban = payment_details.iban

 # To get the swift
 swift = payment_details.swift

 # To get the routing number
 routing_number = payment_details.routing_number

 # To get the account_number
 account_number = payment_details.account_number

</code></pre>

&nbsp;

&nbsp;

Invoice.company_number:&nbsp;&nbsp;List of detected company registration number.

Each object in the list contains an extra attribute:

&nbsp;

<ul>
	<li>type: (String), Generic: VAT NUMBER, TAX ID, GST NUMBER, COMPANY REGISTRATION NUMBER or country specific: TIN (United States), GST/HST (Canada), SIREN/SIRET (France), UEN (Singapore), STNR (Germany), KVK (NL), CIF (Spain), NIF (Portugal), CVR (Denmark), CF (Italy), DIC (Czech Republic), RFC (Mexico), GSTIN (India) ...etc</li>
</ul>

&nbsp;

<pre>
<code class="language-python"># To get the list of company numbers
company_registration_numbers = invoice_data.invoice.company_number

# Loop on each object
for company_number in company_registration_numbers:
 # To get the type of number
 company_number_type = company_number.type

 # To get the company number
 company_number_value = company_number.value</code></pre>

&nbsp;

&nbsp;

<h2>Locale and currency</h2>

&nbsp;

locale: Language ISO code

Contains an extra attribute:

&nbsp;

<ul>
	<li>currency: (String), ISO currency code</li>
</ul>

&nbsp;

<pre>
<code class="language-python"># To get the total language code
language = invoice_data.invoice.locale.value

# To get the invoice currency code
currency = invoice_data.invoice.locale.currency</code></pre>

&nbsp;

How to use extracted invoice fields with Mindee python helper library.

Invoice fields

<h1>Validation checks</h1>

&nbsp;

&nbsp;

Each Invoice object comes with a set of validations checks. it can be accessed under the attribute <code>Invoice.checklist</code>.

&nbsp;

Example:

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(
 invoice_token="your_invoices_api_token_here",
 raise_on_error=True
)

parsed_data = mindee_client.parse_invoice("/path/to/file")

print(parsed_data.invoice.checklist)</code></pre>

&nbsp;

&nbsp;

This sample code should print something like:

&nbsp;

<pre>
<code class="language-bash">{'taxes_match_total_incl': True, 'taxes_match_total_excl': False, 'taxes_plus_total_excl_match_total_incl': True}</code></pre>

&nbsp;

&nbsp;

taxes_match_total_incl: Try to reconstruct the total amount including taxes with taxes line details&nbsp;and compare the result against the extracted total_incl field.

&nbsp;

taxes_match_total_excl: Try to reconstruct the total amount excluding taxes with&nbsp;taxes line details&nbsp;and compare the result against the extracted total_excl field.

&nbsp;

taxes_match_total_incl: Try to reconstruct the total amount including taxes as the sum of taxes line details amount and the total exluding taxes and compare the result against the extracted total_incl field.&nbsp;

&nbsp;

&nbsp;

&nbsp;

<h2>Interpretation: <code>True</code> checks</h2>

&nbsp;

Mindee&#39;s algorithms extract all fields independantly, whatever the type of document. For example, they don&#39;t use the&nbsp;extracted taxes&nbsp;to reconstruct the total amounts. This way, when trying to reconstruct some fields from others with post-processing and comparing them to raw extracted fields, it&#39;s very unlikely to get a matching reconstruction although the predictions were wrong.

&nbsp;

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(invoice_token="your_invoices_api_token_here")

invoice_data = mindee_client.parse_invoice(open("./sample.pdf", 'rb'), input_type="file")

if invoice_data.invoice.checklist["taxes_match_total_incl"] is True:
 # We are sure that taxes and total_incl fields were extracted correctly
 pass

if invoice_data.invoice.checklist["taxes_match_total_excl"] is True:
 # We are sure that taxes and total_excl fields were extracted correctly
 pass

if invoice_data.invoice.checklist["taxes_plus_total_excl_match_total_incl"] is True:
 # We are sure that taxes, total_incl and total_excl fields were extracted correctly
 pass
</code></pre>

&nbsp;

&nbsp;

The&nbsp;<code>mindee</code>&nbsp;library will enforce probabilities to 1 for each field involved in validated check.

&nbsp;

&nbsp;

<h2>Interpretation: <code>False</code> checks</h2>

&nbsp;

The validation checks were done in order to give the user easy way of ensuring maximum precision, but it doesn&#39;t necessarily&nbsp;mean that a <code>False</code>&nbsp;check implies wrong predictions for the involved fields. Here are a few examples leading to impossible checks:

&nbsp;

<ul>
	<li>
	Taxes line details are not always written in invoices, or sometimes they don&#39;t include the tax rate (for example &quot;City Tax&quot;). In this case, as there is no rate extracted by the API and it makes this check impossible to compute.
	</li>
	<li>
	Extra fees with no corresponding taxes are sometimes in the sub-total of the invoice, making this validation impossible.
	</li>
	<li>
	Post total fees (shipping cost ou delivery fees for instance) are excluding from taxes calculation, making the total including impossible to validate.
	</li>
</ul>

&nbsp;

&nbsp;

Built-in invoice data validation rules with Mindee python helper library.

Invoice data validation

<h1>Performances benchmark</h1>

&nbsp;

In a few steps, you can use the mindee python library to perform benchmarks of invoice extraction perfomances against your test dataset.

&nbsp;

<ol>
	<li>Perform api calls for each file in your test set</li>
	<li>Create <code>mindee.Invoice</code> objects from your data (csv, json or whatever)</li>
	<li>Perform the benchmark and analyze the results</li>
</ol>

&nbsp;

&nbsp;

&nbsp;

<h2>1. Perform api calls for each file in your test set</h2>

&nbsp;

Before running the benchmark, we need to collect all the invoice predictions from Mindee API and store them so we can perform the benchmark later without calling the API again.

&nbsp;

To do so, assuming that all your files are stored into a &quot;./test_dataset&quot; folder and that we want to save all the responses into a &quot;./mindee_responses&quot; folder, here is the code:

&nbsp;

<pre>
<code class="language-python">from mindee import Client
import os

TEST_DATA_PATH = "./test_dataset"
RESPONSES_PATH = "./mindee_responses"

mindee_client = Client(invoice_token="your_invoices_api_token_here")

# Loop over all your test files
for test_filename in os.listdir(TEST_DATA_PATH):
 # Get the current file path
 test_file_path = os.path.join(TEST_DATA_PATH, test_filename)
 
 # To make sure we don't stop the process if an error occurs
 try:
 # Parse the current file
 mindee_response = mindee_client.parse_invoice(test_file_path)
 
 # Store the response inside json file to be restored later
 # In this example we use the test file name in the json filename to 
 # be able to retrieve the corresponding file
 response_filepath = os.path.join(RESPONSES_PATH, test_filename+".json")
 mindee_response.dump(response_filepath)
 except Exception as e:
 # In case of error, print the filename so you can understand later
 # what happened
 print(test_filename, e)
</code></pre>

&nbsp;

&nbsp;

&nbsp;

<h2>2. Create <code>mindee.Invoice</code> objects from your data (csv, json,&nbsp;whatever...)</h2>

&nbsp;

The <code>mindee.Invoice</code> class contains a <code>compare()</code> method that takes as inputs two <code>mindee.Invoice</code> objects. Before running our final script, we need now to create a&nbsp;<code>mindee.Invoice object</code> containing the true labels for each fields.

&nbsp;

We&#39;ll use a csv file in this example and the&nbsp;<a href="https://pandas.pydata.org/">pandas</a>&nbsp;library.

&nbsp;

<h3>ground_truth.csv</h3>

<pre>
<code class="language-bash">filename,total_incl,total_excl,invoice_date,invoice_number,due_date,taxes
1.jpg,149.5,115,2020-12-01,F0012020,2021-01-01,11.5-10|23-20</code></pre>

<h3>&nbsp;</h3>

To construct an Invoice object from this dummy csv example, you can simply do:

&nbsp;

<pre>
<code class="language-python">import pandas as pd
from mindee import Invoice

ground_truth_df = pd.read_csv("./ground_truth.csv")


def invoice_from_csv_row(df_row):
 taxes_list = df_row["taxes"].split("|")
 return Invoice(
 total_incl=df_row["total_incl"],
 due_date=df_row["due_date"],
 total_excl=df_row["total_excl"],
 invoice_date=df_row["invoice_date"],
 taxes=[(t.split("-")[0], t.split("-")[1]) for t in taxes_list],
 invoice_number=df_row["invoice_number"]
 )


for index, df_row in ground_truth_df.iterrows():
 invoice_truth = invoice_from_csv_row(df_row)
 print(invoice_truth)
</code></pre>

&nbsp;

&nbsp;

Running this code should print in your console something like this:

&nbsp;

<pre>
<code class="language-bash">-----Invoice data-----
Filename: None 
Invoice number: F0012020 
Total amount including taxes: 149.5 
Total amount excluding taxes: 115.0 
Invoice date: 2020-12-01
Supplier name: None
Taxes: 11.5 10.0%,23.0 20.0%
Total taxes: 34.5
----------------------</code></pre>

&nbsp;

&nbsp;

&nbsp;

<h2>3.&nbsp;Perform the benchmark and see the results</h2>

&nbsp;

Last step, now we need to wrap it all up.

&nbsp;

The Benchmark class has two methods <code>Benchmark.add</code>&nbsp;and <code>Benchmark.save</code>&nbsp;for adding comparison between two Invoice objects, and saving the final metrics.

&nbsp;

&nbsp;

<pre>
<code class="language-python">import pandas as pd
from mindee import Response, Invoice, Benchmark
import os

TEST_DATA_PATH = "./test_dataset"
RESPONSES_PATH = "./mindee_responses"
BENCHMARK_PATH = "./benchmark"

ground_truth_df = pd.read_csv("./ground_truth.csv")
benchmark = Benchmark(BENCHMARK_PATH)


def invoice_from_csv_row(df_row):...
 )


# Loop over each file in our csv
for index, df_row in ground_truth_df.iterrows():
 try:
 # Get test file path
 test_file_path = os.path.join(TEST_DATA_PATH, df_row["filename"])

 # Create ground truth invoice object
 ground_truth_invoice = invoice_from_csv_row(df_row)
 
 # Load the mindee Response for the current file
 mindee_response = Response.load(os.path.join(RESPONSES_PATH, df_row["filename"] + ".json"))

 # Add the comparison between the two invoices to the benchmark
 benchmark.add(
 Invoice.compare(mindee_response.invoice, ground_truth=ground_truth_invoice),
 df_row["filename"]
 )
 
 except Exception as e:
 print(df_row["filename"], e)

benchmark.save()</code></pre>

&nbsp;

&nbsp;

Inside our benchmark folder, you should see a new directory was created, and a metrics.png file shoud have been created inside with the different metrics:

&nbsp;

<img alt="" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2020/10/26/metrics.png" />

&nbsp;

The Invoice benchmark runs on the 7 fields as shown above. For each of them, you get an information of:

accuracy: the proportion of correct predictions

precision: The proportion of correct predictions among all the non null predictions

Benchmark Invoice OCR performances using Mindee python helper library

Performances benchmark

python

<h1>How to setup your Mindee account</h1>

&nbsp;

&nbsp;

Want to try Mindee APIs? You have already signed up and want to make your first API calls? You&#39;re in the right place !

&nbsp;

&nbsp;

<h2>Signup / login</h2>

&nbsp;

Go to the&nbsp;<a href="https://platform.mindee.net/signup">Signup page</a>&nbsp;to create your account. No credit card is required. You can start using the APIs right away.

&nbsp;

You already have an account ? Go to the&nbsp;<a href="https://platform.mindee.net/login">Login page</a>. If you forgot your password, click&nbsp;<a href="https://platform.mindee.net/password-forgotten">the link</a> and follow the instructions. Check your spam if you don&#39;t receive the email.

&nbsp;

&nbsp;

<h2>Subscribe to an API</h2>

&nbsp;

Once logged in, you land on our API store.

&nbsp;

<img alt="" src="https://mindee-public-website.s3.amazonaws.com/blog/2020/06/11/expense_receipt_card.jpg" />

&nbsp;

Click on the desired API card to subscribe. All our APIs have a free plan.

&nbsp;

&nbsp;

<h2>You are set to go !</h2>

&nbsp;

At this point, everything is set up to use Mindee APIs.

&nbsp;

You can either :&nbsp;

<ol>
	<li>Go to the <a href="https://mindee.com/documentation/get-started/platform-tour">Live Interface</a>&nbsp;to drag and drop images and instantly see the results</li>
	<li>Go to the <a href="https://mindee.com/documentation/get-started/platform-tour">Documentation section</a> and make your first call through the API</li>
</ol>

Set up your account

<h1>Platform tour</h1>

&nbsp;

&nbsp;

<h2>Dashboard</h2>

&nbsp;

From the dashboard section, monitor your API usage.

&nbsp;

&nbsp;

<img alt="" src="https://mindee-public-website.s3.amazonaws.com/blog/2020/06/10/dashboard_article.jpg" />

&nbsp;

Metrics :

 
Traffic: number of inputs processed (images and pdf pages).

&nbsp;

Errors:&nbsp;number of HTTP errors.

&nbsp;

Latency: average server-side computation time in seconds.

&nbsp;

Plan: your current monthly plan. Your default subscription&nbsp;is a free plan that lasts forever and automatically renews each month. Click on the &quot;change plan&quot; button to adjust your subscription according to your needs.&nbsp;While on,&nbsp;your plan will automatically be renewed each month.

&nbsp;

Quota:&nbsp;remaining number of images or pdf pages you can process under your current monthly subscription. 
&nbsp;

Invoicing: find here all your past invoices.

&nbsp;

&nbsp;

<h2>Documentation</h2>

&nbsp;

Everything you need to use the API in your apps.

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/05/19/documentation_article.jpg" />

&nbsp;

HTTP request: expected HTTP request format.

&nbsp;

Features: specify the JSON response fields, data types&nbsp;and formats.

&nbsp;

Sample code: ready to use sample codes in different languages.&nbsp;Copy and paste in your environment.

&nbsp;

Usage: details about supported input files and technical limitations.

&nbsp;

JSON response: a model JSON response filled with dummy data to navigate more easily the response structure.

&nbsp;

&nbsp;

<h2>Credentials</h2>

&nbsp;

Manage your API access.

&nbsp;

<img alt="" src="https://mindee-public-website.s3.amazonaws.com/blog/2020/06/10/credentials_article.jpg" />

&nbsp;

&nbsp;

API keys : create, revoke and rename your API keys from there. Note that revokation is permanent and cannot be undone.

&nbsp;

&nbsp;

&nbsp;

<h2>Live interface</h2>

&nbsp;

This is your home for no code testing. Drag and drop your own documents to see the API in action.

&nbsp;

Quickly visualize all the extracted fields along with the raw API JSON response.

&nbsp;

<img alt="" src="https://mindee-public-website.s3.amazonaws.com/blog/2020/06/10/live_interface_article.jpg" />

Platform tour

<h1>Make your first request</h1>

&nbsp;

&nbsp;

Prerequisites:&nbsp;for the following, you&#39;ll need a Mindee account and a live API subscription. You can go learn how to create an account and freely subscribe to an API <a href="https://mindee.com/documentation/get-started/setup-your-account">here</a>.

&nbsp;

&nbsp;

<h2>Create an API key</h2>

&nbsp;

Go to the &ldquo;credentials&rdquo; section of your API. Click on &ldquo;add key&rdquo; to create a new token for this API.

&nbsp;

<img alt="" src="https://mindee-public-website.s3.amazonaws.com/blog/2020/06/10/new_token_article.jpg" />

&nbsp;

&nbsp;

&nbsp;

<h2>Get your sample code</h2>

&nbsp;

Go to the Documentation section and find the sample codes in the top right corner:

&nbsp;

<img alt="" src="https://mindee-public-website.s3.amazonaws.com/blog/2020/06/10/sample_code_article.jpg" />

&nbsp;

&nbsp;

&nbsp;

<h2>Call the API</h2>

&nbsp;

The&nbsp;sample codes will help you&nbsp;perform the POST request to Mindee APIs in different languages, and send your inputs the right way.&nbsp;

&nbsp;

For all the sample codes, replace &quot;mindee_api_url&quot; and &quot;your_token_here&quot; placeholders with your own credentials. Everything is available in your API documentation.

&nbsp;

Python

Make sure you have the <a href="https://requests.readthedocs.io/en/master/">requests</a> package installed.&nbsp;If not, you can install it in your env by running :&nbsp;

&nbsp;

<pre>
<code class="language-bash">pip install requests</code></pre>

&nbsp;

<pre>
<code class="language-python">import requests

url = "mindee_api_url"

with open("/path/to/my/file", "rb") as myfile:
 files = {"file": myfile}
 headers = {"X-Inferuser-Token": "your_token_here"}
 response = requests.post(url, files=files, headers=headers)
 print(response.text)</code></pre>

&nbsp;

&nbsp;

cURL

&nbsp;

<pre>
<code>curl -X POST \
 mindee_api_url \
 -H 'X-Inferuser-Token: your_token_here' \
 -H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
 -F file=@/path/to/your/file.png</code></pre>

&nbsp;

&nbsp;

Javascript

Load the following code in an html file and open it with your web browser. The page displays a button allowing you to upload an image. Once uploaded, go check your console, the response is there.

&nbsp;

<pre>
<code class="language-html">&lt;form onsubmit="mindeeSubmit(event)" &gt;
 &lt;input type="file" id="my-file-input" name="file" /&gt;
 &lt;input type="submit" /&gt;
&lt;/form&gt;

&lt;script type="text/javascript"&gt;
const mindeeSubmit = (evt) =&gt; {
 evt.preventDefault()
 let myFileInput = document.getElementById('my-file-input');
 let myFile = myFileInput.files[0]
 if (!myFile) { return }
 let data = new FormData();
 data.append("file", myFile, myFile.name);

 let xhr = new XMLHttpRequest();

 xhr.addEventListener("readystatechange", function () {
 if (this.readyState === 4) {
 console.log(this.responseText);
 }
 });

 xhr.open("POST", "mindee_api_url");
 xhr.setRequestHeader("X-Inferuser-Token", "your_token_here");
 xhr.send(data);
}
&lt;/script&gt;</code></pre>

&nbsp;

You&nbsp;get something like this ;&nbsp;

&nbsp;

<img alt="" src="https://mindee-public-website.s3.amazonaws.com/blog/2020/06/10/html_article.jpg" />

&nbsp;

Ruby

&nbsp;

<pre>
<code class="language-ruby">require 'uri'
require 'net/http'
require 'net/https'
require 'mime/types'

url = URI("mindee_api_url")
file = "/path/to/your/file.png"

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
request = Net::HTTP::Post.new(url)
request["content-type"] = 'multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW'
request["X-Inferuser-Token"] = 'your_token_here'
request.body = "------WebKitFormBoundary7MA4YWxkTrZu0gW\r\nContent-Disposition: form-data; name=\"file\"; filename=\"#{File.basename(file)}\"\r\nContent-Type: #{MIME::Types.type_for(file)}\r\n\r\n------WebKitFormBoundary7MA4YWxkTrZu0gW--"

response = http.request(request)
puts response.read_body</code></pre>

&nbsp;

Node.js

Make sure you have the&nbsp;<a href="https://www.npmjs.com/package/request">request</a>&nbsp;package installed before running this code.&nbsp;If not, you can install it in your env by running :&nbsp;

&nbsp;

<pre>
<code class="language-bash">npm install request</code></pre>

&nbsp;

<pre>
<code class="language-javascript">const fs = require("fs");
const request = require("request");
const options = {
	method: "POST",
	url: "mindee_api_url",
	headers: {
		"X-Inferuser-Token": "your_token_here",
	},
	formData : {
		"file" : fs.createReadStream("/path/to/your/file.png")
	}
};
request(options, function (err, res, body) {
	if(err) console.log(err);
	console.log(body);
});</code></pre>

&nbsp;

How to make your first request to Mindee APIs

Make your first request

Get started

<h1>&nbsp;Base URL</h1>

&nbsp;

&nbsp;

Prediction APIs are served through a unique endpoint depending on the product you select. They follow this pattern :

&nbsp;

&nbsp;

<pre>
<code class="language-http">https://api.mindee.net/products/&lt;api_name&gt;/&lt;api_version&gt;/predict</code></pre>

&nbsp;

&nbsp;

You need to specify an &lt;api_name&gt; and a &lt;api_version&gt; that corresponds to the API you want to use. You can find what you need in the specific API documentation, see <a href="https://mindee.com/documentation/get-started/platform-tour" style="text-decoration:none">platform tour documentation</a> for more details.

&nbsp;

&nbsp;

<img alt="" src="https://mindee-public-website.s3.amazonaws.com/blog/2020/06/10/base_url_1_article.jpg" />

&nbsp;

<h2>About versionning</h2>

&nbsp;

We release new versions of&nbsp;our APIs for two&nbsp;reasons :&nbsp;

&nbsp;

<ul>
	<li>minor changes: &nbsp;we often release new minor changes of our APIs with new models (faster and&nbsp; / or better), without any loss of service. These releases are transparent on your side as the response structure is untouched.</li>
</ul>

&nbsp;

<ul>
	<li>major changes: this is a major iteration, with possibly no retrocompatibility of the JSON response. Those versions can include new locales, change in the way predictions are formatted... You can simultaneously use several major versions with the same subscription and tokens. You can switch between versions in the dropdown at the top of the page when inside your subscription API area.</li>
</ul>

Base URL

<h1>Authentication</h1>

&nbsp;

You must include a valid token through an HTTP custom header to authenticate your API calls :

&nbsp;

&nbsp;

<pre>
<code>'X-Inferuser-Token: &lt;my-token-here&gt;'</code></pre>

&nbsp;

&nbsp;

For each API, you can create as many tokens as you want in the credentials page. See <a href="https://mindee.com/documentation/get-started/platform-tour" style="text-decoration:none">platform tour documentation </a>for more details.

&nbsp;

&nbsp;

<h2>Token revokation</h2>

&nbsp;

&nbsp;

You can revoke a token on the credentials section. Note that once a token is revoked, you can&#39;t create a new token with the same name, and using this token to authenticate to your API will lead to a 401 error.

&nbsp;

<img alt="" src="https://mindee-public-website.s3.amazonaws.com/blog/2020/06/10/revoke_authentication_article.jpg" />

&nbsp;

<h2>Technical limitations</h2>

&nbsp;

Note that technical limitations are set up for each account depending on your plan. The default limitations are low, even for the free plans, but if you need to lower those limitations, feel free to ask the support team.

&nbsp;

<img alt="" src="https://mindee-public-website.s3.amazonaws.com/blog/2020/06/10/usage_authentication_article.jpg" />

&nbsp;

These limitations are configured for your entire subscription, regardless of tokens or IP address.

How to authenticate your requests to Mindee APIs

Authentication

&nbsp;&nbsp;

<h1>&nbsp;Managing errors</h1>

&nbsp;

Requests can fail for two main reasons:

&nbsp;

Network issues:&nbsp;the server could not be reached

Application error:&nbsp;the server rejected the request

&nbsp;

In the latter case, the error reported by the API&nbsp;contains:

&nbsp;

<pre>
<code class="language-json">{
 "details": "...",
 "message": "optional"
}</code></pre>

&nbsp;

&nbsp;

<table border="1" cellpadding="1" cellspacing="1">
	<thead>
		<tr>
			<th scope="col">Code</th>
			<th scope="col">Possible reasons</th>
		</tr>
	</thead>
	<tbody>
		<tr>
			<td>401 (Unauthorized)</td>
			<td>
			Bad token:&nbsp;you didn&rsquo;t provide the authentication header or the token is wrong.
			</td>
		</tr>
		<tr>
			<td>400 (Bad request)</td>
			<td>
			Wrong file type: file types allowed:&nbsp; png, jpg, webp, pdf

			&nbsp;

			Missing input file: see&nbsp;<a href="https://mindee.com/documentation/apis-overview/input-data">How to manage input files</a>&nbsp;documentation

			&nbsp;

			Too many pages in pdf:&nbsp;APIs have a maximum limit of pages allowed for pdf files

			&nbsp;

			File is too big: APIs have a maximum file size allowed
			</td>
		</tr>
		<tr>
			<td>404 (Not found)</td>
			<td>
			Wrong version: wrong version in base URL
			</td>
		</tr>
		<tr>
			<td>429&nbsp;(Too many requests)</td>
			<td>
			Plan limit reached: you have reached your maximum number of requests.

			&nbsp;

			Too high&nbsp;frequency: number of requests over time is limited. See <a href="https://mindee.com/documentation/apis-overview/authentication">Authentication documentation</a>
			</td>
		</tr>
	</tbody>
</table>

Managing errors

<h1>Input data</h1>

&nbsp;

&nbsp;

Input files must be sent&nbsp;using&nbsp;multipart/form-data encoding with the whole binary file or stream file object, base64 are not supported yet. To see how you can make it work in your language, you can refer to&nbsp;<a href="https://mindee.com/documentation/get-started/make-your-first-request">Make your first request</a>.

&nbsp;

&nbsp;

<h2>Images</h2>

&nbsp;

We support JPG, PNG and WEBP format. Note that our APIs have a maximum file size limitation, think about downsizing your input images if you manipulate heavy files. Always prefer JPG over PDF for scans and photos.

&nbsp;

Tips for images :&nbsp;

&nbsp;

<ol>
	<li>Reduce too high resolution: downscale your image to have a faster upload time and processing time. Because the text can be very small sometimes, resizing the image too much will make it impossible to read. The rule of thumb is that big images should be resized close to 3 megapixels.</li>
	<li>Keep the aspect ratio: when resizing an image, always keep the original aspect ratio.</li>
	<li>Do not upscale: never upscale a low resolution image on your side! This will decrease the algorithm accuracy. It is best to avoid very low resolution images anyhow.</li>
	<li>Do not preprocess images: it is unnecessary to transform your image in black and white, change brightness / contrast if the image looks fine.</li>
	<li>Ask the chat!: If you have any question related to your image input data, feel free to ask the chat to talk to an expert.</li>
</ol>

&nbsp;

<h2>&nbsp;</h2>

<h2>Working with PDFs</h2>

&nbsp;

Our APIs support multi-page PDF files as well. Because we need to convert the PDF pages to actual images, the processing time may be superior compared to images processing.

&nbsp;

Tips for PDFs :&nbsp;

&nbsp;

<ol>
	<li>To be used only if you must: always prefer JPG/PNG format instead of PDF when possible. This is especially true when handling photos and scans. PDF files should be reserved for digital documents only for maximum performance.</li>
	<li>Working with several pages: &nbsp;your PDF may contain several pages. In this scenario, we will handle each page as a separate document. You will therefore receive as many predictions as there are pages. See <a href="https://mindee.com/documentation/apis-overview/json-response-scheme">JSON response scheme</a> documentation for more details.</li>
	<li>Limitations: here is a maximum number of pages you can send, check the Documentation page of your selected API.</li>
</ol>

Input data best practices for Mindee APIs

Input data

<h1>JSON response scheme</h1>

&nbsp;

Each API response follows the same response scheme. The predictions content is specific with the API you use. Once you have subscribed to your preferred API, read the documentation to check for all the outputs available.

&nbsp;

<pre>
<code class="language-json">{
 "call": {
 "endpoint": {
 "name": "endpoint", 
 "version": "2.1"
 },
 "finished_at": "2017-04-11T02:30:49+00:00", 
 "id": "2a81a403-0389-4b39-9b1f-22b5febf795e", 
 "n_documents": 1, 
 "n_inputs": 1, 
 "processing_time": 0.725, 
 "started_at": "2017-04-11T02:30:49+00:00"
 }, 
 "documents": [
 {
 "id": "bf53e6b1-1d29-4187-97ec-bab10eca5744", 
 "name": "filename.png"
 }
 ],
 "predictions": [

 ...

 ]
}
</code></pre>

&nbsp;

call:&nbsp;metadata about the request

&nbsp;

id: a UUID - this identifies your http call

processing_time: server side processing time in seconds

started_at: datetime for when the call has been received

finished_at: datetime the call has been fully processed

n_documents:&nbsp; number of documents processed.

n_inputs: number of images and/or pdf pages processed. Your quota will be decreased by this value.

endpoint:

&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; name: name of the endpoint used to make the call

&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; version: selected API version

&nbsp;

&nbsp;

documents&nbsp;:&nbsp;array with the details of each document sent to the API. NB: a pdf page counts for one document.

id: a UUID - this identifies your document

name: name of file sent, for a pdf this may add the page number at the end

&nbsp;

predictions: Array with the predictions for each document processed. The structure inside each element of this array is specific with the API you use. NB: for now, each pdf page will output a prediction

General JSON response scheme for Mindee APIs

JSON response scheme

APIs overview

<h1>Parsing receipts with Mindee&#39;s API</h1>

&nbsp;

Anyone who has filed an expense report can tell you: receipt tracking and expense logging is a headache.&nbsp; Enter Mindee&rsquo;s receipt parsing API, which uses deep learning to automatically, accurately and instantaneously parse your receipt details.

&nbsp;

In this tutorial, we will walk through the steps to use Mindee&rsquo;s Receipt Parsing API.&nbsp; Let&rsquo;s get started!&nbsp;

&nbsp;

<h2>API Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free Mindee account. <a href="https://platform.mindee.net/signup" target="_blank">Sign up</a>&nbsp;and confirm your email to login.</li>
	<li>A receipt.&nbsp; Look in your bag/wallet for a recent one, or do a <a href="https://www.google.com/search?q=cafe+receipt" target="_blank">Google Image search</a> for a receipt and download a few to test with.</li>
</ol>

&nbsp;

&nbsp;

&nbsp;<img src="https://lh3.googleusercontent.com/Dlw1124fbpDBiZamK8ohelvR9MNK0UlnKutB9UDTfp4vD6FzNN26sXwlkJUFGhiTNvAW_PxWNFz98dGSicLm8qkrEey7VVBY1AE_bErDOKLILhnFGL3UzEmvVetBfad-70qmSXLG" />

&nbsp;

<h2>Setting up the API</h2>

&nbsp;

Log into you Mindee account and access your Expense Receipt API environment by clicking the Expense Receipts card:

&nbsp;

<img src="https://lh6.googleusercontent.com/7V0dnGLFfrvOHViat-kxpTs888k2C3Nk1AJlSlDGks9U5tJD2DjWXXMzC0ks-BcmfRR4-gfKILmhtLesNoMrdfRBjb6OmySZALeqeuUNTBEDnLc_87Kc-vkRYbVKhGjmh4bTKQx9" />

&nbsp;

To activate the API, click the &ldquo;Try for Free&rdquo; button to access our generous free tier. You&rsquo;ll land on the dashboard page - where you can quickly see API usage (you have none right now, but that will change).&nbsp; On the left navigation, there are links to &ldquo;Documentation&rdquo;, &ldquo;Credentials&rdquo; and &ldquo;Live Interface&rdquo;.&nbsp; The docs tab has all of the technical details you&rsquo;ll need to build for the receipts API endpoint, and the Live Interface is a cool interactive demo. Rather than try out the demo, we want to build with the API,&nbsp; so click on &ldquo;Credentials&rdquo; to create an API token.

&nbsp;

Add a new token. In this example, I&rsquo;ve named it &ldquo;Tutorial&rdquo;:

&nbsp;

<img src="https://mindee-public-website.s3.amazonaws.com/blog/2020/09/01/pasted-image-0-1.png" style="height:247px; width:500px" />

&nbsp;

Click &ldquo;Add New Key&rdquo; and you&rsquo;ll be able to see your API token,.

&nbsp;

Now, we are ready to make an API call.&nbsp; In this example, we&rsquo;ll be using cURL.

&nbsp;

<pre>
<code class="language-bash">curl -X POST \ https://api.mindee.net/products/expense_receipts/v2/predict \ 

-H 'X-Inferuser-Token: {apiToken}’ \

-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \

-F file=@/path/to/your/file.png</code></pre>

&nbsp;

Simply replace {apiToken}&nbsp;with your new API token and /path/to/your/file/png with the path to your receipt.&nbsp;

NOTE: You can also copy this code right from the documentation tab of the API with your API token inserted for you.

&nbsp;

In this example, I used a receipt from the grocery store in Koln airport (my last business trip in 2020):

&nbsp;

<img src="https://mindee-public-website.s3.amazonaws.com/blog/2020/09/03/comporessed_doug_receipt.jpg" style="height:717px; width:500px" />

&nbsp;

Pasting the cURL sample into my terminal, I hit enter and about a second later, I received a JSON response with the receipt details.&nbsp; The full JSON can be accessed <a href="https://mindee-public-website.s3.amazonaws.com/blog/2020/09/01/rewe.json">here</a>.&nbsp; Since the response is quite verbose, we will walk through the various fields section by section.

&nbsp;

&nbsp;

<h2>API Response: Parsing Results</h2>

&nbsp;

<h3>Summary &amp; Documents section:</h3>

&nbsp;

The first two sections of the response contain information about the API call made:

&nbsp;

<pre>
<code class="language-json">"call": {
    "endpoint": {
        "name": "expense_receipts",
        "version": "2.1"
    },
    "finished_at": "2020-08-29T18:01:22+00:00",
    "id": "e47d8654-0df7-4839-a282-2c04bf293886",
    "n_documents": 1,
    "n_inputs": 1,
    "processing_time": 1.087,
    "started_at": "2020-08-29T18:01:21+00:00"
},
"documents": [
    {
    "id": "66d9adc6-76cf-4c42-8622-3dadb660ac32",
    "name": "IMG_20200301_073354.jpg"
    }
],</code></pre>

&nbsp;

The call section tells us that we ran on the expense receipts endpoint, uploading one document that is one page long, and after just about 1 second, the file was processed.&nbsp; The documents section gives the Mindee id for the file, and the filename.

&nbsp;

<h2>Predictions:&nbsp;&nbsp;</h2>

&nbsp;

Now we are getting to the exciting stuff.&nbsp; The Predictions section is broken into several sections.&nbsp; Several of these are identifying fields on the receipt, and others are using Machine Learning to deduce information from the receipt.&nbsp; Let&rsquo;s go through each section:

<h3>&nbsp;</h3>

<h3>Category</h3>

&nbsp;

<pre>
<code class="language-json">category": {
     "probability": 0.51,
     "value": "miscellaneous"
},</code></pre>

&nbsp;

The API make a prediction on the type of purchase.&nbsp; In this case, it is 51% sure it is miscellaneous.&nbsp; The possible categories are [toll, food, parking, transport, accommodation, gasoline, miscellaneous].

&nbsp;

<h3>Date</h3>

&nbsp;

Identified from text on&nbsp;the receipt and converted into ISO format. This purchase was made on February 3, 2020, and the model is 99% confident in that choice.&nbsp; The segmentation bounding box provides 4 (x,y) coordinates indicating where the date was pulled from the receipt [(0,0) is the upper left corner, (1,1) is the bottom right corner].

&nbsp;

<pre>
<code class="language-json">"date": {
    "iso": "2020-02-03",
    "probability": 0.99,
    "raw": "03-02-2020",
    "segmentation": {
    "bounding_box": [
      [0.64,0.661],
      [0.801,0.661],
      [0.801,0.686],
      [0.64,0.686]
  ]
}</code></pre>

&nbsp;

<h3>Locale</h3>

&nbsp;

Using data from the&nbsp;receipt, the API can predict where the purchase was made, the language and the currency:. Check the documentation for the latest support. At the time of writing, support is centered on Europe and North America.

&nbsp;

<pre>
<code class="language-json">"locale": {
    "country": "DE",
    "currency": "EUR",
    "language": "de",
    "probability": 0.77,
    "value": "de-DE"
},</code></pre>

&nbsp;

In the case of my receipt, it is 77% confident that the purchase is in German, made in Germany, and in euros.

&nbsp;

<h3>Merchant</h3>

&nbsp;

<pre>
<code class="language-json">"merchant": {
    "name": "REWE",
    "probability": 0.91,
    "segmentation": {
    "bounding_box": [
      [0.279, 0.135],
      [0.719, 0.135],
      [0.719,0.23],
      [0.279,0.23]
     ]
}</code></pre>

The API correctly predicted (it was 91% sure) that it was a REWE store. Again, four (x,y) points mark the location of the text naming the Merchant on the image.

&nbsp;

<h3>Orientation</h3>

&nbsp;

<pre>
<code class="language-json">"orientation": {
   "degrees":0,
   "probability": 0.99
},</code></pre>

Did the document require rotation before parsing?&nbsp; Measured in 90 degree increments [0.90.180.270]. In&nbsp;this case, it did not require any rotation.

&nbsp;

<h3>Taxes</h3>

&nbsp;

<pre>
<code class="language-json">"taxes": [],</code></pre>

&nbsp;

If any taxes are identified in the receipt, the will appear here.&nbsp; In this case, no taxes&nbsp;were found (but this is the correct result).

<h2>&nbsp;</h2>

<h3>Time</h3>

&nbsp;

<pre>
<code class="language-json">"time": {
    "iso": "15:50",
    "probability": 0.99,
    "raw": "15:50",
    "segmentation": {
    "bounding_box": [
      [0.649,0.898],
      [0.732,0.898],
      [0.732,0.925],
      [0.649,0.925]
    ]
}</code></pre>

&nbsp;

Time the receipt was printed, confidence, and the (x,y) coordinates that bound the field in the image.

<h2>&nbsp;</h2>

<h3>Total</h3>

&nbsp;

Perhaps the most important part of the receipt, the total spent, along with confidence and the box indicating the location on the receipt.

&nbsp;

<pre>
<code class="language-json">"total": {
    "amount": 17.74,
    "probability": 0.99,
    "segmentation": {
    "bounding_box": [
     [0.663,0.589],
     [0.765,0.589],
     [0.765,0.617],
     [0.663,0.617]
   ]
}</code></pre>

&nbsp;

<h2>&nbsp;</h2>

<h2>Summary:</h2>

&nbsp;

In just over 1 second, a receipt was uploaded, parsed and the response returned to the end user.&nbsp; We know that &euro;17.74 was spent at a REWE in Germany on February 3, 2020 at 15:50.&nbsp; Using the bounding boxes, we can have the user validate the values, and then input this data into an expense management system.&nbsp;&nbsp;

&nbsp;

&nbsp;

<h2>Conclusion</h2>

&nbsp;

Using the MIndee receipt parsing API, you can quickly validate receipts, allowing for faster, more accurate (and less painful) expense management for our users. If you have questions, please reach out to us in the chat widget in the bottom right.

Anyone who has filed an expense report can tell you: receipt tracking and expense logging is a headache.  Enter Mindee’s receipt parsing API, which uses deep learning to automatically, accurately and instantaneously parse your receipt details. In this tutorial, we will walk through the steps to use Mindee’s Receipt Parsing API.  Let’s get started!

<h1>Extract receipt data with Mindee&#39;s API using python</h1>

&nbsp;

In this tutorial, you will learn how to parse a receipt image using python. This can help you automatically extract&nbsp;data from receipts without human interaction and use it in your backend applications for many accounting purposes.

&nbsp;

If you want to get more details about how to use the mindee python library:

<ul>
	<li>1.&nbsp;<a href="https://mindee.com/documentation/apis/tutorial-parsing-receipts-with-mindees-api/python/getting-started">Python SDK overview</a></li>
	<li>2.&nbsp;<a href="https://mindee.com/documentation/apis/tutorial-parsing-receipts-with-mindees-api/python/receipt-fields">Extracted receipt fields</a></li>
	<li>3.&nbsp;<a href="https://mindee.com/documentation/apis/tutorial-parsing-receipts-with-mindees-api/python/save-and-restore">Save and restore Receipt objects</a></li>
	<li>4.&nbsp;<a href="https://mindee.com/documentation/apis/tutorial-parsing-receipts-with-mindees-api/python/receipt-data-validation">Receipt data validations</a></li>
	<li>5.&nbsp;<a href="https://mindee.com/documentation/apis/tutorial-parsing-receipts-with-mindees-api/python/performances-benchmark">Benchmark Receipt OCR performances</a></li>
</ul>

&nbsp;

<h2>Tutorial</h2>

&nbsp;

The final script can be used within any python middleware application or REST API serving your frontend interfaces.

&nbsp;

In the end of this tutorial, we&rsquo;ll show you how to highlight the extracted receipt fields on the image using the open source computer vision library OpenCV.

&nbsp;

The final result looks like this:

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/09/03/ezgifcom-video-to-gif-14.gif" />

&nbsp;

&nbsp;

<h2>API Prerequisites</h2>

&nbsp;

<ol>
	<li>You&rsquo;ll need a free Mindee account. <a href="https://platform.mindee.net/signup" target="_blank">Sign up</a>&nbsp;and confirm your email to login.</li>
	<li>A receipt.&nbsp; Look in your bag/wallet for a recent one, download one&nbsp;<a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2020/11/04/sample_receipt-35.jpg">here</a> &nbsp;or do a <a href="https://www.google.com/search?q=cafe+receipt" target="_blank">Google Image search</a> for a receipt and download a few to test with.</li>
</ol>

&nbsp;

<h2>&nbsp;</h2>

<h2>Setup the project</h2>

&nbsp;

Create an empty directory wherever you want on your laptop, we&rsquo;ll call ours &ldquo;python_receipt&rsquo;.

 
We recommend you create a virtual environment before going to the next step, in order to avoid installing the required packages globally on your machine. To do so, run the following command:

&nbsp;

(macOS and Linux)

<pre>
<code class="language-bash">python -m venv venv</code></pre>

 
(windows)

<pre>
<code class="language-bash">py -m venv venv</code></pre>

&nbsp;

Then, you&#39;ll need to activate your venv, so that pip will install packages in that folder.

&nbsp;

(macOS and Linux)

<pre>
<code class="language-bash">source venv/bin/activate</code></pre>

 
(Windows)

<pre>
<code class="language-bash">.\env\Scripts\activate</code></pre>

&nbsp;

&nbsp;

We need now to Install the mindee python library&nbsp;from PyPi using pip, a package manager for Python.

&nbsp;

<pre>
<code class="language-bash">pip install mindee</code></pre>

&nbsp;

If the install throws an error, you might need to upgrade your version of pip. To do so, run:

&nbsp;

<pre>
<code class="language-bash">pip install --upgrade pip </code></pre>

&nbsp;

Don&#39;t have pip installed? Try installing it, by running this from the command line:

&nbsp;

<pre>
<code class="language-bash">$ curl https://bootstrap.pypa.io/get-pip.py | python</code></pre>

&nbsp;

That&rsquo;s it for the setup, let&rsquo;s call the API.

&nbsp;

&nbsp;

<h2>Call the Mindee receipt API</h2>

&nbsp;

Log in to the <a href="https://platform.mindee.net">platform</a> and click on the following item to access the Expense receipts API:

&nbsp;

<img alt="Expense receipt API card" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/09/03/expense_receipt_card.jpg" style="height:280px; width:300px" />

&nbsp;

If you don&rsquo;t have an API token yet for this API, go to the &ldquo;credentials&rdquo; section and create a new API Key.&nbsp;

&nbsp;

<img alt="new api token" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/09/03/new_token.jpg" style="height:146px; width:300px" />

&nbsp;

Create a &ldquo;main.py&rdquo; file in your project directory, and paste the following sample code:

&nbsp;

<pre>
<code class="language-python">from mindee import Client

filepath = "/path/to/my/file"
expense_receipt_token = "my-token-here"

mindee_client = Client(expense_receipt_token=expense_receipt_token)


if __name__ == "__main__":
 
 response = mindee_client.parse_receipt(filepath)

 print(response.receipt)</code></pre>

&nbsp;

&nbsp;

Replace the &ldquo;path/to/my/file&rdquo; placeholder in the code with the path of the receipt image you want to extract data from.

&nbsp;

Replace the &ldquo;my-token-here&rdquo; with the API token you created previously in the platform.

&nbsp;

You can get back to your console and run the script

&nbsp;

<pre>
<code class="language-python">python main.py</code></pre>

&nbsp;

You should see the details of the receipt printed in your console:

&nbsp;

<pre>
<code class="language-bash">-----Receipt data-----
Filename: receipt_.jpg
Total amount: 10.2 
Date: 2016-02-26
Category: food
Time: 15:20
Merchant name: CLACHAN
Taxes: 1.7; 20.0%; _
Total taxes: 1.7
----------------------</code></pre>

&nbsp;

Now let&rsquo;s take a look at the Receipt object.

&nbsp;

<h2>&nbsp;</h2>

<h2>Extracted receipt data</h2>

&nbsp;

The API extracts a list of different fields in the receipt (total amount, taxes, date &hellip;).&nbsp;

&nbsp;

The Receipt&nbsp;object under the&nbsp;<code>Response.receipt</code>&nbsp;attribute has all the features you need to get the receipt data and coordinates in the image to highlight the results.

&nbsp;

Here is a piece of code to show you how to get some of them:

&nbsp;

<pre>
<code class="language-python">from mindee import Client

filepath = "/path/to/my/file"
expense_receipt_token = "my-token-here"

mindee_client = Client(expense_receipt_token=expense_receipt_token)


if __name__ == "__main__":
 
 response = mindee_client.parse_receipt(filepath)

 # To get the total amount including taxes value (float), ex: 14.24
 print(response.receipt.total_incl.value)
 
 # To get the total amount excluding taxes value (float), ex: 10.21
 print(response.receipt.total_excl.value)
 
 # To get the total tax amount value (float), ex: 8.42
 print(response.receipt.total_tax.value)
 
 # To get the list of taxes
 print(response.receipt.taxes)

 # Loop on each Tax field
 for tax in response.receipt.taxes:
 # To get the tax amount
 print(tax.value)
 
 # To get the tax code for from a tax object
 print(tax.code)
 
 # To get the tax rate
 print(tax.rate)
 
 # To get the receipt date (string)
 print(response.receipt.date.value)
 
 # To get the receipt merchant name (string)
 print(response.receipt.merchant_name.value)
 
 # To get the receipt time (string)
 print(response.receipt.time.value)</code></pre>

&nbsp;

Run the script and check out the results in your console!

&nbsp;&nbsp;

Now we are going to add a piece of code that will highlight the features extracted on the pdf. It can help you or your users to very quickly validate the data extraction.

&nbsp;

&nbsp;

<h2>Highlight features on the image</h2>

&nbsp;

Let&rsquo;s try to highlight the features as if someone did it with a pen.

&nbsp;

First, you&rsquo;ll need to install the computer vision python library OpenCV if you don&rsquo;t have it already installed in your env. To do so, run:

&nbsp;

&nbsp;

<pre>
<code>pip install opencv-python</code></pre>

&nbsp;

Each field extracted in the&nbsp;<code>Response.receipt</code>&nbsp;object has a&nbsp;<code>Field.bbox</code>&nbsp;object. It&#39;s a list of vertices in relative coordinates representing the bounding box (% of the image width and height).

&nbsp;

Check&nbsp;what&#39;s inside for the total_incl.bbox for example, doing:

&nbsp;

<pre>
<code class="language-python">print(response.receipt.total_incl.bbox)</code></pre>

&nbsp;

Now that we know how to access the bounding boxes coordinates, we are going to create our&nbsp;highlighter function in which we can feed any field bounding boxes&nbsp;in different steps, as follows:

&nbsp;

<ol>
	<li>Check if the file is a pdf or an image and load it as an OpenCV image in both scenarios</li>
	<li>Create a mask image</li>
	<li>Loop on each feature coordinates and draw the feature rectangle on our mask</li>
	<li>Overlay the mask and original image with alpha</li>
	<li>Display image to the user</li>
</ol>

&nbsp;

Note: each coordinate returned by the API is relative (in % of the image). You&#39;ll see there is a relative to absolute conversion in the code.

&nbsp;

Here is the code step by step:

&nbsp;

<pre>
<code>def highlight_features(img_path, coordinates):
 # step 1: Loads the image as an openCV image
 cv_image = cv2.imread(img_path)

 # step 2: create mask image
 overlay = cv_image.copy()
 h, w = cv_image.shape[:2]

 # step 3: Loop on each feature coordinates and draw the feature rectangle on our mask
 for coord in coordinates:
 if len(coord):
 # Create points absolute coordinates
 pt1 = (int(w*coord[0][0]), int(h*coord[0][1]))
 pt2 = (int(w*coord[2][0]), int(h*coord[2][1]))
 cv2.rectangle(overlay, pt1, pt2, (70, 230, 244), cv2.FILLED)

 # step 4: Overlay the mask and original image with alpha
 final_image = cv2.addWeighted(overlay, 0.5, cv_image, 0.5, 0)

 # step 5: Display image to the user
 cv2.imshow("highlighted_image", cv2.resize(final_image, (600, int(600*h/w))))
 cv2.waitKey(0)</code></pre>

 
&nbsp;

Finally, we just have to modify&nbsp;the code for&nbsp;executing the highlighting function and printing the results. Here is&nbsp;the final code:

&nbsp;

&nbsp;

<pre>
<code class="language-python">from mindee import Client
import cv2


def highlight_features(img_path, coordinates):
 # step 1: Load the file as a numpy image 
 cv_image = cv2.imread(img_path)

 # step 2: create mask image
 overlay = cv_image.copy()
 h, w = cv_image.shape[:2]

 # step 3: Loop on each feature coordinates and draw the feature rectangle on our mask
 for coord in coordinates:
 if len(coord):
 pt1 = (int(w*coord[0][0]), int(h*coord[0][1]))
 pt2 = (int(w*coord[2][0]), int(h*coord[2][1]))
 cv2.rectangle(overlay, pt1, pt2, (70, 230, 244), cv2.FILLED)

 # step 4: Overlay the mask and original image with alpha
 final_image = cv2.addWeighted(overlay, 0.5, cv_image, 0.5, 0)

 # step 5: Display image to the user
 cv2.imshow("highlighted_image", cv2.resize(final_image, (600, int(600*h/w))))
 cv2.waitKey(0)


filepath = "/path/to/my/file"
expense_receipt_token = "my-token-here"

mindee_client = Client(expense_receipt_token=expense_receipt_token)


if __name__ == "__main__":
 
 response = mindee_client.parse_receipt(filepath)

 print(response.receipt)

 highlight_features(
 filepath,
 [
 response.receipt.total_incl.bbox,
 response.receipt.time.bbox,
 response.receipt.merchant_name.bbox,
 response.receipt.date.bbox
 ]
 )
</code></pre>

 
&nbsp;

And the final result!

&nbsp;

<img alt="" src="https://mindee-public-website.s3.amazonaws.com/blog/2020/11/05/highlghted_image.jpg" />

&nbsp;

<h2>Conclusion</h2>

&nbsp;

In just over 1 second, a receipt was uploaded, the API extracted the receipt data, then the result was&nbsp;parsed&nbsp;and&nbsp;highlighted in&nbsp;the image. Cool right?

&nbsp;

If you wanted to use this kind of script to display results to your users, I&#39;d advise you to do the highlighting in the front-end application, as sending images back from your middleware is not the best option because of payload sizes. The other solution would be to store the final image using cv2.imwrite(...) but it would make your client download the result. To help you with client-side rendering for your images, you can check out our&nbsp;<a href="https://github.com/publicMindee/react-mindee-js">open-source ReactJS SDK</a>.

&nbsp;

If you have questions, please reach out to us in the chat widget in the bottom right.

This tutorial explains how to extract data from receipt image using python and Mindee"s API, then how to parse the result and highlight the different features on the source image.

<h1>Getting started</h1>

<h2>&nbsp;</h2>

&nbsp;

Get started with receipt parsing in python.

&nbsp;

<h2>1. Install the mindee python helper library</h2>

&nbsp;

Install from PyPi using&nbsp;<a href="https://pip.pypa.io/en/latest/" rel="nofollow">pip</a>, a package manager for Python.

&nbsp;

<pre>
<code class="language-bash">pip install mindee</code></pre>

&nbsp;

Don&#39;t have pip installed? Try installing it, by running this from the command line:

&nbsp;

<pre>
<code class="language-bash">$ curl https://bootstrap.pypa.io/get-pip.py | python</code></pre>

&nbsp;

Getting started with the Mindee API couldn&#39;t be easier. Create a&nbsp;<code>Client</code>&nbsp;and you&#39;re ready to go.

&nbsp;

&nbsp;

&nbsp;

<h2>2. Instantiate your Client</h2>

&nbsp;

The&nbsp;<code>mindee.Client</code>&nbsp;needs your API credentials. You can either pass these directly to the constructor (see the code below) or via environment variables.

&nbsp;

Depending on what type of document you want to parse, you need to add specifics auth token for each endpoint.

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(
 expense_receipt_token="your_expense_receipts_api_token_here",
 raise_on_error=True
)</code></pre>

&nbsp;

<ul>
	<li>expense_receipt_token:&nbsp;(string) API key for expense_receipts endpoint</li>
	<li>raise_on_error: (boolean, default True) Specify wheter or not raising an Exception when HTTP errors occur. If set to False, when calling the parse_receipt method, the <code>Response.receipt</code> object returned will have be set to None and the&nbsp; <code>Response.receipts</code>&nbsp;list will be empty.</li>
</ul>

&nbsp;

&nbsp;

We suggest storing your credentials as environment variables. Why? You&#39;ll never have to worry about committing your credentials and accidentally posting them somewhere public.

&nbsp;

&nbsp;

&nbsp;

<h2>3. Parse a receipt</h2>

&nbsp;

Using the parse_receipt&nbsp;method from your client object, you can pass any image or pdf file to get the receipt data.

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(
 expense_receipt_token="your_expense_receipts_api_token_here",
 raise_on_error=True
)

parsed_data = mindee_client.parse_receipt("/path/to/file")
</code></pre>

&nbsp;

&nbsp;

<h3>Input types</h3>

&nbsp;

You can pass your input file in three ways:

&nbsp;

From file path

<pre>
<code class="language-python">receipt_data = mindee_client.parse_receipt('/path/to/file', input_type="path")</code></pre>

&nbsp;

&nbsp;From a file object

<pre>
<code class="language-python">with open('/path/to/file', 'rb') as fp:
 receipt_data = mindee_client.parse_receipt(fp, input_type="file")
</code></pre>

&nbsp;

From a base64

<pre>
<code class="language-python">receipt_data = mindee_client.parse_receipt(base64_string, input_type="base64")</code></pre>

&nbsp;

&nbsp;

receipt_data structure

&nbsp;

The receipt_data object returned by the&nbsp;<code>parse_receipt</code>&nbsp;method contains the following elements:

&nbsp;

receipt_data.receipt

&nbsp;

The receipt attribute&nbsp;is the Receipt object constructed by gathering all the pages into a single document. The method used for creating a single receipt object with multiple pages relies on fields confidence scores. Basically, we iterate over each page and for each field, we keep the one that has the highest probability.

&nbsp;

<pre>
<code class="language-python">receipt_data.receipt # returns a unique object from class Receipt</code></pre>

<h4>&nbsp;</h4>

&nbsp;

receipt_data.receipts

&nbsp;

For multi pages pdf, the receipts attribute contains one receipt object per input. For a single-page pdf or an image, the list will only contain 1 item, and for multi-page pdf, it will contain one item per page.

&nbsp;

<pre>
<code class="language-python">receipt_data.receipts # [Receipt, Receipt ...] </code></pre>

<h4>&nbsp;</h4>

&nbsp;

receipt_data.http_response

&nbsp;

Contains the full Mindee API HTTP response object in JSON format

&nbsp;

<pre>
<code class="language-python">receipt_data.http_response # full HTTP request object </code></pre>

&nbsp;

&nbsp;

&nbsp;

<h2>4. Display the results</h2>

&nbsp;

You only have to print your receipt object to display the different extracted fields:

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(
 expense_receipt_token="your_expense_receipts_api_token_here",
 raise_on_error=True
)

receipt_data = mindee_client.parse_receipt("/path/to/receipt")

print(receipt_data.receipt)</code></pre>

&nbsp;

&nbsp;

Getting started with Mindee receipt OCR for parsing receipt image using python.

<h1>Receipt fields</h1>

&nbsp;

&nbsp;

<h2>General fields structure</h2>

&nbsp;

Each receipt object contains a set of different fields.

&nbsp;

To access a receipt object, you need to create a <code>mindee.Client&nbsp;</code>and call the <code>Client.parse_receipt</code>&nbsp;method:

&nbsp;

<pre>
<code>from mindee import Client

# Instantiate your client
mindee_client = Client(expense_receipts_token="your_token_here")

# Call the Mindee expense receipts endpoint and parse the result
receipt_data = mindee_client.parse_receipt("/path/to/my/file")</code></pre>

&nbsp;

Whatever the field type, each of them contains the four following attributes:

&nbsp;

<ul>
	<li>Value: (Str or Float depending on the field type), corresponds to the field value. Set to None if the field was not extracted.</li>
	<li>Probability: (Float), confidence score of the field prediction.</li>
	<li>bbox:(Array[Float]), contains the relative vertices coordinates of the bounding box containing the field in the image. If the field is not written, the bbox is an empty array.&nbsp;</li>
	<li>reconstructed: (Bool), True if the field was reconstructed using other fields.</li>
</ul>

&nbsp;

Depending on the Field type, there can be extra attributes.

&nbsp;

&nbsp;

&nbsp;

<h2>Total amounts</h2>

&nbsp;

&nbsp;receipt.total_incl:&nbsp;Total amount including taxes

<pre>
<code class="language-python"># To get the total amount including taxes value (float), ex: 14.24
total_incl = receipt_data.receipt.total_incl.value
</code></pre>

&nbsp;

&nbsp;receipt.total_excl:&nbsp;Total amount excluding taxes

<pre>
<code class="language-python"># To get the total amount excluding taxes value (float), ex: 10.21
total_excl = receipt_data.receipt.total_excl.value</code></pre>

&nbsp;

receipt.total_tax: Total tax value reconstructed from tax lines

<pre>
<code class="language-python"># To get the total tax amount value (float), ex: 8.42
total_tax = receipt_data.receipt.total_tax.value</code></pre>

<img src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2020/10/26/image-20201026171243-1.gif" />

&nbsp;

&nbsp;

<h2>Taxes</h2>

&nbsp;

receipt.taxes:&nbsp;Array of Tax fields

Each of the tax field has two extra attributes:

&nbsp;

<ul>
	<li>code: (String), Optional tax code. (HST, GST... for canadian; City Tax, State tax for US etc..)</li>
	<li>rate: (Float), Optional tax rate.</li>
</ul>

&nbsp;

<pre>
<code class="language-python"># To get the list of taxes
taxes = receipt_data.receipt.taxes

# Loop on each Tax field
for tax in taxes:
 # To get the tax amount
 tax_amount = tax.value

 # To get the tax code for from a tax object
 tax_code = tax.code
 
 # To get the tax rate
 tax_rate = tax.rate</code></pre>

&nbsp;

&nbsp;

&nbsp;

<h2>Dates</h2>

&nbsp;

Each date field comes with an extra attribute:

&nbsp;

<ul>
	<li>date_object: (Datetime), datetime object from python datetime.date library</li>
</ul>

&nbsp;

&nbsp;

&nbsp;receipt.date: Payment date related to the receipt

<pre>
<code class="language-python"># To get the receipt date of issuance (string)
receipt_date = receipt_data.receipt.value</code></pre>

&nbsp;

&nbsp;

<h2>Supplier informations</h2>

&nbsp;

&nbsp;receipt.supplier:&nbsp;Supplier name as written in the receipt&nbsp;(logo)

<pre>
<code class="language-python"># To get the supplier name
supplier_name = receipt_data.receipt.supplier.value</code></pre>

&nbsp;

&nbsp;

&nbsp;

<h2>Locale and currency</h2>

&nbsp;

locale: Language ISO code

Contains an extra attribute:

&nbsp;

<ul>
	<li>currency: (String), ISO currency code</li>
</ul>

&nbsp;

<pre>
<code class="language-python"># To get the total language code
language = receipt_data.receipt.locale.value

# To get the receipt currency code
currency = receipt_data.receipt.locale.currency</code></pre>

&nbsp;

&nbsp;

<h2>&nbsp;Category</h2>

&nbsp;

category: Receipt category among the list:&nbsp; toll, food, parking, transport, accommodation, gasoline, miscellaneous

<pre>
<code class="language-python"># To get the category
category = receipt_data.receipt.category.value</code></pre>

&nbsp;

&nbsp;

<h2>&nbsp;Time</h2>

&nbsp;

time: Time of purchase with 24 hours formatting (hh:mm)

<pre>
<code class="language-python"># To get the category
category = receipt_data.receipt.time.value</code></pre>

&nbsp;

&nbsp;

How to use extracted receipt data with Mindee python helper library.

Receipt fields

<h1>Save and restore</h1>

&nbsp;

The&nbsp;<code>mindee.Client</code>&nbsp;includes methods for saving and restoring api calls. Those methods are helpful if you want to perform tests or if you simply want to reload receipts objects corresponding to your files without performing any additional request.

&nbsp;

&nbsp;

<h2>Save a call</h2>

&nbsp;

The <code>mindee.Response</code> object returned by the <code>mindee.Client</code> when parsing a file is responsible for wrapping all the data from Mindee API calls. It&nbsp;includes a <code>dump</code> method that can be used as follows:

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(expense_receipts_token="your_expense_receipts_api_token_here")

receipt_data = mindee_client.parse_receipt("/path/to/your/file", input_type="file")

receipt_data.dump("./receipt_dump.json")</code></pre>

&nbsp;

&nbsp;

<h2>Restore a call</h2>

&nbsp;

To restore a client response object, you can use the <code>mindee.Response</code> as follows:

&nbsp;

<pre>
<code class="language-python">from mindee import Response

receipt_data = Response.load("./receipt_dump.json")</code></pre>

&nbsp;

&nbsp;

Save and restore Mindee expense receipt API response with Mindee python helper library

<h1>Validation checks</h1>

&nbsp;

&nbsp;

Each receipt object comes with a set of data validation controls. it can be accessed under the attribute <code>Receipt.checklist</code>.

&nbsp;

Example:

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(
 expense_receipts_token="your_expense_receipts_api_token_here",
 raise_on_error=True
)

parsed_data = mindee_client.parse_receipt("/path/to/file")

print(parsed_data.receipt.checklist)</code></pre>

&nbsp;

&nbsp;

This sample code should print something like:

&nbsp;

<pre>
<code class="language-bash">{'taxes_match_total_incl': True}</code></pre>

&nbsp;

&nbsp;

taxes_match_total_incl: Try to reconstruct the total amount including taxes with taxes line details&nbsp;and compare the result against the extracted total_incl field.

&nbsp;

<h2>Interpretation: <code>True</code> checks</h2>

&nbsp;

Mindee&#39;s algorithms extract all fields independantly, whatever the type of document. For example, they don&#39;t use the&nbsp;extracted taxes&nbsp;to reconstruct the total amounts. This way, when trying to reconstruct some fields from others with post-processing and comparing them to raw extracted fields, it&#39;s very unlikely to get a matching reconstruction although the predictions were wrong.

&nbsp;

&nbsp;

<pre>
<code class="language-python">from mindee import Client


mindee_client = Client(
 expense_receipts_token="your_expense_receipts_api_token_here",
 raise_on_error=True
)

receipt_data = mindee_client.parse_receipt("path/to/file")

if receipt_data.receipt.checklist["taxes_match_total_incl"] is True:
 # We are sure that taxes and total_incl fields were extracted correctly
 pass
</code></pre>

&nbsp;

&nbsp;

The&nbsp;<code>mindee</code>&nbsp;library will enforce probabilities to 1 for each field involved in validated check.

&nbsp;

Built-in receipt data validation rules with Mindee python helper library.

Receipt data validation

<h1>Performances benchmark</h1>

&nbsp;

In a few steps, you can use the mindee python library to perform benchmarks of receipt OCR API perfomances against your test dataset.

&nbsp;

<ol>
	<li>Perform api calls for each file in your test set</li>
	<li>Create <code>mindee.Receipt</code> objects from your data (csv, json or whatever)</li>
	<li>Perform the benchmark and analyze the results</li>
</ol>

&nbsp;

&nbsp;

&nbsp;

<h2>1. Perform api calls for each file in your test set</h2>

&nbsp;

Before running the benchmark, we need to collect all the receipt predictions from Mindee API and store them so we can perform the benchmark later without calling the API again.

&nbsp;

To do so, assuming that all your files are stored into a &quot;./test_dataset&quot; folder and that we want to save all the responses into a &quot;./mindee_responses&quot; folder, here is the code:

&nbsp;

<pre>
<code class="language-python">from mindee import Client
import os


TEST_DATA_PATH = "./test_dataset"
RESPONSES_PATH = "./mindee_responses"


mindee_client = Client(expense_receipts_token="your_expense_receipts_api_token_here")


# Loop over all your test files
for test_filename in os.listdir(TEST_DATA_PATH):
 
 # Get the current file path
 test_file_path = os.path.join(TEST_DATA_PATH, test_filename)
 
 # To make sure we don't stop the process if an error occurs
 try:
 # Parse the current file
 mindee_response = mindee_client.parse_receipt(test_file_path)
 
 # Store the response inside json file to be restored later
 # In this example we use the test file name in the json filename to 
 # be able to retrieve the corresponding file
 response_filepath = os.path.join(RESPONSES_PATH, test_filename+".json")
 mindee_response.dump(response_filepath)

 except Exception as e:
 # In case of error, print the filename so you can understand later
 # what happened
 print(test_filename, e)
</code></pre>

&nbsp;

&nbsp;

&nbsp;

<h2>2. Create <code>mindee.Receipt</code> objects from your data (csv, json,&nbsp;whatever...)</h2>

&nbsp;

The <code>mindee.Receipt</code> class contains a static&nbsp;<code>compare()</code> method that takes as inputs two <code>mindee.Receipt</code> objects. Before running our final script, we need now to create a&nbsp;<code>mindee.Receipt object</code> containing the true labels for each fields.

&nbsp;

We&#39;ll use a csv file in this example and the&nbsp;<a href="https://pandas.pydata.org/">pandas</a>&nbsp;library.

&nbsp;

<h3>ground_truth.csv</h3>

<pre>
<code class="language-bash">filename,total_incl,date,taxes
1.jpg,149.5,2020-12-01,11.5-10|23-20</code></pre>

<h3>&nbsp;</h3>

To construct a Receipt&nbsp;object from this dummy csv example, you can simply do:

&nbsp;

<pre>
<code class="language-python">import pandas as pd
from mindee import Receipt

ground_truth_df = pd.read_csv("./ground_truth.csv")


def receipt_from_csv_row(df_row):
 taxes_list = df_row["taxes"].split("|")
 return Receipt(
 total_incl=df_row["total_incl"],
 date=df_row["date"],
 taxes=[(t.split("-")[0], t.split("-")[1]) for t in taxes_list]
 )


for index, df_row in ground_truth_df.iterrows():
 receipt_truth = receipt_from_csv_row(df_row)
 print(receipt_truth)
</code></pre>

&nbsp;

&nbsp;

Running this code should print receipt data from your csv file in your console.

&nbsp;

&nbsp;

&nbsp;

<h2>3.&nbsp;Perform the benchmark and see the results</h2>

&nbsp;

Last step, now we need to wrap it all up.

&nbsp;

The Benchmark class has two methods <code>Benchmark.add</code>&nbsp;and <code>Benchmark.save</code>&nbsp;for adding comparison between two receipt objects, and saving the final metrics.

&nbsp;

&nbsp;

<pre>
<code class="language-python">import pandas as pd
from mindee import Response, Receipt, Benchmark
import os

TEST_DATA_PATH = "./test_dataset"
RESPONSES_PATH = "./mindee_responses"
BENCHMARK_PATH = "./benchmark"

ground_truth_df = pd.read_csv("./ground_truth.csv")
benchmark = Benchmark(BENCHMARK_PATH)


def receipt_from_csv_row(df_row):
 # Your method for creating the Receipt ground truth
 pass
 

# Loop over each file in our csv
for index, df_row in ground_truth_df.iterrows():
 try:
 # Get test file path
 test_file_path = os.path.join(TEST_DATA_PATH, df_row["filename"])

 # Create ground truth receipt object
 ground_truth_receipt = receipt_from_csv_row(df_row)
 
 # Load the mindee Response for the current file
 mindee_response = Response.load(os.path.join(RESPONSES_PATH, df_row["filename"] + ".json"))

 # Add the comparison between the two receipts to the benchmark
 benchmark.add(
 Receipt.compare(mindee_response.receipt, ground_truth=ground_truth_receipt),
 df_row["filename"]
 )
 
 except Exception as e:
 print(df_row["filename"], e)

benchmark.save()</code></pre>

&nbsp;

&nbsp;

Inside our benchmark folder, you should see a new directory was created, and a metrics.png file shoud have been created inside with the different metrics:

&nbsp;

<img alt="" src="https://mindee-public-website.s3.amazonaws.com/blog/2020/11/05/metrics-1.png" />

&nbsp;

The receipt benchmark runs on the 7 fields as shown above. For each of them, you get an information of:

accuracy: the proportion of correct predictions

precision: The proportion of correct predictions among all the non null predictions

Benchmark Receipt OCR performances using Mindee python helper library

<h1>Extract receipt data with Mindee&rsquo;s API using reactjs</h1>

&nbsp;

In this tutorial, you&#39;ll learn how to create an optimized and interactive user interface for automatic receipt data extraction using ReactJS and Mindee. We are going to use both Mindee API and open source <a href="https://github.com/publicMindee/react-mindee-js">ReactJS SDK</a>. The SDK has a lot of features built-in such as shapes visualisation and interactions, zoom, multi pages ... It will help&nbsp;you create any dynamic user experience based on the receipt image&nbsp;to help your users validate very quickly the results predicted by the API.

&nbsp;

In 10 minutes,&nbsp;you&#39;ll get something like this:

&nbsp;

<img alt="Mindee receipt parsing react js" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/09/04/ezgifcom-optimize-2.gif" />

&nbsp;

Let&#39;s get started!

&nbsp;

<h2>&nbsp;</h2>

<h2>API Prerequisites</h2>

&nbsp;

<ol>
	<li>We are using create-react-app framework for bootstrapping the app. If you don&#39;t have it installed, you can do it&nbsp;<a href="https://create-react-app.dev/docs/getting-started/">here</a>. You can also use another React framework for creating the app but some code may differ from what is written in this tutorial.</li>
	<li>You&rsquo;ll need a free Mindee account. <a href="https://platform.mindee.net/signup" target="_blank">Sign up</a>&nbsp;and confirm your email to login.</li>
	<li>A receipt.&nbsp; Look in your bag/wallet for a recent one, or do a <a href="https://www.google.com/search?q=cafe+receipt" target="_blank">Google Image search</a> for a receipt and download a few to test with.</li>
</ol>

&nbsp;

&nbsp;

<h2>Setup your project</h2>

&nbsp;

First, we need to create a new react application using create-react-app. Open up your terminal and go to the directory you want to create your project in, and create your app with the name you want (we call ours receipt_parsing):

&nbsp;

<pre>
<code class="language-bash">npx create-react-app receipt_parsing</code></pre>

&nbsp;

To make sure everything went well, go to your directory and run the app

&nbsp;

<pre>
<code class="language-bash">cd receipt_parsing
yarn start</code></pre>

&nbsp;

Or using npm

&nbsp;

<pre>
<code class="language-bash">cd receipt_parsing
npm run start</code></pre>

&nbsp;

Your browser is going to open a new window and run the app.

&nbsp;

A last package install before we can start coding! As we are going to use Mindee ReactJS SDK, we need to install it. It&#39;s open-source and you can find all the documentation on the <a href="https://github.com/publicMindee/react-mindee-js">Github repo</a>. To install in your React app, just run in your terminal:

&nbsp;

<pre>
<code class="language-bash">yarn add react-mindee-js</code></pre>

&nbsp;

Or using npm:

&nbsp;

<pre>
<code class="language-bash">npm install react-mindee-js</code></pre>

&nbsp;

Now, using your favourite IDE, open the App.js file that was created by create-react-app, inside the src folder of your project directory. In the whole tutorial, we are going to work only on this file. Once you access the App.js file in your terminal, delete everything and just paste this code:

&nbsp;

<pre>
<code class="language-ruby">import React, {useState} from 'react';

import {
 AnnotationViewer,
 createPolygonFromCoordinates
} from "react-mindee-js";

const HelloWorld = () =&gt; {
 const [image, setImage] = useState(null)

 const shapes = [
 {
 polygon: createPolygonFromCoordinates([[0.66, 0.3], [0.12, 0.5], [0.52, 0.8], [0.82, 0.24]]),
 color: "#fd3246",
 data:"This is red shape",
 index: 0,
 active: {
 color: '#00f0f0',
 lineWidth: 3
 }
 }
 ]

 return (
 &lt;div&gt;
 &lt;input type="file" onChange={e =&gt; setImage(URL.createObjectURL(e.target.files[0]))}/&gt;
 {
 image &amp;&amp; &lt;div style={{ height: '500px', width: '400px'}}&gt;
 &lt;AnnotationViewer
 shapes={shapes}
 image={image}
 onShapeClick={shape =&gt; console.log('shapeClicked', shape)}
 onShapeHover={shape =&gt; console.log('shapeHovered', shape)}
 /&gt;
 &lt;/div&gt;
 }
 &lt;/div&gt;
 );
}
export default HelloWorld;</code></pre>

&nbsp;

We define a polygon by its [x,y] coordinates and colors.&nbsp; When an image is added from the form, we call the onChange to apply the polygon, and write to the console on Click and Hover actions.

You&#39;re all setup to start. When you run the app, you should see a&nbsp;file input button on your web page, and once you upload an image using it, you get something like:

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/09/04/hello_world_receipt_react.jpg" />

The ploygon is defined but the four [x,y] vertices, and the color is red, but on hover, it turns blue (00f0f0).

<h2>&nbsp;</h2>

<h2>&nbsp;</h2>

<h2>Parse the receipt</h2>

&nbsp;

Log&nbsp;onto the <a href="https://platform.mindee.net/login">platform</a> and get into your Expense receipt API environment by clicking the following card.

<img alt="Expense receipt API card" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/09/03/expense_receipt_card.jpg" />

&nbsp;

If you don&rsquo;t have an API token yet for this API, go to the &ldquo;credentials&rdquo; section and create a new API Key.&nbsp;

&nbsp;

<img alt="new api token" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/09/03/new_token.jpg" />

&nbsp;

Inside your App.js file, replace everything with this code, and replace the &#39;my-token-here&#39; placeholder with the one you just created on the platform:

&nbsp;

<pre>
<code class="language-javascript">import React, {useState} from 'react';
import {
 AnnotationViewer,
 formatPrediction
} from "react-mindee-js";

const HelloWorld = () =&gt; {
 const [image, setImage] = useState(null)
 const [shapes, setShapes] = useState([])

 const parseReceipt = (e) =&gt; {
 let myFile = e.target.files[0]

 if (!myFile) {return}
 
 setImage(URL.createObjectURL(myFile))

 let data = new FormData();
 data.append("file", myFile, myFile.name);

 let xhr = new XMLHttpRequest();

 xhr.addEventListener("readystatechange", function () {
 if (this.readyState === 4) {
 let json_response = JSON.parse(this.responseText)
 setShapes(formatPrediction(json_response.predictions[0]))
 }
 });

 xhr.open("POST", "https://api.mindee.net/products/expense_receipts/v2/predict");
 xhr.setRequestHeader("X-Inferuser-Token", "my-token-here");
 xhr.send(data);
 }

 return (
 &lt;div&gt;
 &lt;input type="file" onChange={e =&gt; parseReceipt(e)}/&gt;
 {
 image &amp;&amp; &lt;div style={{ height: '500px', width: '400px'}}&gt;
 &lt;AnnotationViewer
 shapes={shapes}
 image={image}
 onShapeClick={shape =&gt; console.log('shapeClicked', shape)}
 onShapeHover={shape =&gt; console.log('shapeHovered', shape)}
 /&gt;
 &lt;/div&gt;
 }
 &lt;/div&gt;
 );
}
export default HelloWorld;</code></pre>

&nbsp;

In this iteration, we have replaced the onChange callback function to&nbsp;a new function parseReceipt. Basically, this function will call the Receipt parsing API (and send the image to Mindee to extract the receipt details), and transform the results to very easily generate shapes using formatPrediction() function from the SDK.

&nbsp;

Run the app, you should see something like this when you upload a receipt image:

&nbsp;

<img alt="react mindee parse receipt" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/09/04/hello_world_receipt_react_step.jpg" />

&nbsp;

All boxes displayed on the image are interractive and you can bind custom onClick and onMouseOver mouse events functions to them. You can also try to zoom in the image, try to move the image inside the viewer etc.. All features are explained on the Github repo.

&nbsp;

Important note: Calling the receipt parsing API client-side is not&nbsp;secure, as your API token is present in the browser data, potentially exposing this to the world.&nbsp; This approach should only be used for&nbsp;prototyping,&nbsp;you&#39;ll need a back-end middleware to make a secure production app.

&nbsp;

Now we are just going to change a few lines of codes to display the data extracted by the API. To do so, we are going to add a new state object &#39;receiptData&#39;, and display a few features on the web page. In your code, change the HelloWorld component to this (don&#39;t forget to fix your api token):

&nbsp;

<pre>
<code class="language-javascript">const HelloWorld = () =&gt; {
 const [image, setImage] = useState(null)
 const [shapes, setShapes] = useState([])
 const [receiptData, setReceiptData] = useState(null)

 const parseReceipt = (e) =&gt; {
 let myFile = e.target.files[0]

 if (!myFile) {
 return
 }
 setImage(URL.createObjectURL(myFile))

 let data = new FormData();
 data.append("file", myFile, myFile.name);

 let xhr = new XMLHttpRequest();

 xhr.addEventListener("readystatechange", function () {
 if (this.readyState === 4) {
 let json_response = JSON.parse(this.responseText)
 setShapes(formatPrediction(json_response.predictions[0]))
 setReceiptData(json_response.predictions[0])
 }
 });

 xhr.open("POST", "https://api.mindee.net/products/expense_receipts/v2/predict");
 xhr.setRequestHeader("X-Inferuser-Token", "my-token-here");
 xhr.send(data);
 }

 return (
 &lt;div&gt;
 &lt;input type="file" onChange={e =&gt; parseReceipt(e)}/&gt;
 {
 image &amp;&amp; &lt;&gt;
 &lt;div style={{height: '500px', width: '400px'}}&gt;
 &lt;AnnotationViewer
 shapes={shapes}
 image={image}
 /&gt;
 &lt;/div&gt;
 {
 receiptData &amp;&amp;
 &lt;div style={{marginLeft: "50px"}}&gt;
 &lt;p&gt;Total amount : {receiptData.total.amount}&lt;/p&gt;
 &lt;p&gt;Date: {receiptData.date.iso}&lt;/p&gt;
 &lt;p&gt;Time: {receiptData.time.iso}&lt;/p&gt;
 &lt;p&gt;Category: {receiptData.category.value}&lt;/p&gt;
 &lt;/div&gt;
 }
 &lt;/&gt;
 }
 &lt;/div&gt;
 );
}</code></pre>

&nbsp;

Run the app, upload a receipt, you should get:

&nbsp;

<img alt="Reactjs mindee parsing receipt" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/09/04/hello_world_receipt_react_step.jpg" />

&nbsp;

We&#39;re almost done!

&nbsp;

<h2>Bind interractions between images boxes and corresponding receipt text features</h2>

&nbsp;

To make sure your user can validate very easily data extracted by the API, the ReactJS SDK allows you to add interactions with visual features on the images and the rest of your app.

&nbsp;

First, we&#39;ll create a function that change the isActive attribute inside a shape object, from it&#39;s name. The SDK will understand this attribute and will modify the shape layout (you can modify the active layout by changing colors and line width in the active attribute of a shape).

&nbsp;

<pre>
<code class="language-javascript">const highlightFeature = (featureName) =&gt; {
 let newShapes = []
 shapes.forEach(shape =&gt; {
 shape.isActive=(shape.featureName === featureName)
 newShapes.push(shape)
 }
 )
 setShapes(newShapes)
}</code></pre>

&nbsp;

We&#39;ll then modify a bit the rendered code, and bind on the onMouseOver event of the text features the highlight function.

&nbsp;

<pre>
<code class="language-javascript">return (
 &lt;div&gt;
 &lt;input type="file" onChange={e =&gt; parseReceipt(e)}/&gt;
 {
 image &amp;&amp; &lt;&gt;
 &lt;div style={{height: '500px', width: '400px'}}&gt;
 &lt;AnnotationViewer
 shapes={shapes}
 image={image}
 /&gt;
 &lt;/div&gt;
 {
 receiptData &amp;&amp;
 &lt;div style={{marginLeft: "50px"}}&gt;
 &lt;p onMouseOver={() =&gt; highlightFeature('total')}&gt;Total amount : {receiptData.total.amount}&lt;/p&gt;
 &lt;p onMouseOver={() =&gt; highlightFeature('date')}&gt;Date: {receiptData.date.iso}&lt;/p&gt;
 &lt;p onMouseOver={() =&gt; highlightFeature('time')}&gt;Time: {receiptData.time.iso}&lt;/p&gt;
 &lt;p&gt;Category: {receiptData.category.value}&lt;/p&gt;
 &lt;/div&gt;
 }
 &lt;/&gt;
 }
 &lt;/div&gt;
 );</code></pre>

&nbsp;

Final App.js

<pre>
<code class="language-javascript">import React, {useState} from 'react';
import './App.css';

import {
 AnnotationViewer,
 formatPrediction
} from "react-mindee-js";

const HelloWorld = () =&gt; {
 const [image, setImage] = useState(null)
 const [shapes, setShapes] = useState([])
 const [receiptData, setReceiptData] = useState(null)

 const parseReceipt = (e) =&gt; {
 let myFile = e.target.files[0]

 if (!myFile) {
 return
 }
 setImage(URL.createObjectURL(myFile))

 let data = new FormData();
 data.append("file", myFile, myFile.name);

 let xhr = new XMLHttpRequest();

 xhr.addEventListener("readystatechange", function () {
 if (this.readyState === 4) {
 let json_response = JSON.parse(this.responseText)
 setShapes(formatPrediction(json_response.predictions[0]))
 setReceiptData(json_response.predictions[0])
 }
 });

 xhr.open("POST", "https://api.mindee.net/products/expense_receipts/v2/predict");
 xhr.setRequestHeader("X-Inferuser-Token", "my-token-here");
 xhr.send(data);
 }

 const highlightFeature = (featureName) =&gt; {
 let newShapes = []
 shapes.forEach(shape =&gt; {
 shape.isActive=(shape.featureName === featureName)
 newShapes.push(shape)
 }
 )
 setShapes(newShapes)
 }

 return (
 &lt;div&gt;
 &lt;input type="file" onChange={e =&gt; parseReceipt(e)}/&gt;
 {
 image &amp;&amp; &lt;&gt;
 &lt;div style={{height: '500px', width: '400px'}}&gt;
 &lt;AnnotationViewer
 shapes={shapes}
 image={image}
 /&gt;
 &lt;/div&gt;
 {
 receiptData &amp;&amp;
 &lt;div style={{marginLeft: "50px"}}&gt;
 &lt;p onMouseOver={() =&gt; highlightFeature('total')}&gt;Total amount : {receiptData.total.amount}&lt;/p&gt;
 &lt;p onMouseOver={() =&gt; highlightFeature('date')}&gt;Date: {receiptData.date.iso}&lt;/p&gt;
 &lt;p onMouseOver={() =&gt; highlightFeature('time')}&gt;Time: {receiptData.time.iso}&lt;/p&gt;
 &lt;p&gt;Category: {receiptData.category.value}&lt;/p&gt;
 &lt;/div&gt;
 }
 &lt;/&gt;
 }
 &lt;/div&gt;
 );
}
export default HelloWorld;</code></pre>

&nbsp;

Running your app, you should see now that when you hover over a text line at the bottom,&nbsp;the color of the box highlighting the value on the receipt&nbsp;will change. You can modify the colours as you want to make the change more or less dramatic, as you please.&nbsp; All of the details for this is&nbsp;explained in the Github readme.

&nbsp;

<h2>Conclusion</h2>

&nbsp;

Using our ReactJS library makes identifying regions in the image easy.&nbsp; The Mindee Receipt Parsing API extracts the relevant details (and the location in the image). Then the react library highlights the locations with the AnnotationViewer.&nbsp;Combined, you have a powerful tool to extract data from images, and display them for verification to your users.

&nbsp;

&nbsp;

Tutorial on how to extract data from a receipt image through a reactjs user interface. You will learn how to parse the different data extracted by the Mindee API and how to create a beautiful user interface using Mindee ReactJS library.

reactjs

<h1>Extract receipt data with Mindee&rsquo;s API using NodeJS</h1>

&nbsp;

To get started with receipt parsing using Mindee API in node.js, you can use our node helper library;

&nbsp;

<ul>
	<li>1.&nbsp;<a href="https://mindee.com/documentation/apis/tutorial-parsing-receipts-with-mindees-api/parse-receipts-nodejs/getting-started">Node.js SDK Overview</a></li>
	<li>2.&nbsp;<a href="https://mindee.com/documentation/apis/tutorial-parsing-receipts-with-mindees-api/parse-receipts-nodejs/receipt-fields">Extracted receipt fields</a></li>
</ul>

&nbsp;

If you want to see how to use the API using HTTP requests, you can follow this guide.

&nbsp;

&nbsp;

<h2>Tutorial</h2>

&nbsp;

In&nbsp;this example, we will build from scratch, a NodeJS based web application that accepts a receipt image or PDF, and returns extracted data from the receipt.

&nbsp;

The code for the sample code is availble on <a href="https://github.com/dougsillars/mindee_node_receipt">Github</a>, and also available on <a href="https://glitch.com/edit/#!/grove-tree-sprite">Glitch</a>.&nbsp; For simplicity, we&#39;ll follow along in the Glitch example.

&nbsp;

<div class="youtube-embed-wrapper" style="height:0; overflow:hidden; padding-bottom:56.25%; padding-top:30px; position:relative"><iframe frameborder="0" height="360" src="https://www.youtube.com/embed/3r5HhjShNtk" style="position:absolute;top:0;left:0;width:100%;height:100%" width="640"></iframe></div>

&nbsp;

&nbsp;

<h2>Glitch example</h2>

&nbsp;

In the server.js, we need to initialise all of our required modules:

&nbsp;

<pre>
<code class="language-javascript">const express = require('express');
const app = express();
const pug = require('pug');

const path = require('path');
const fs = require("fs");
const axios = require('axios');
let FormData = require('form-data');
//formidable takes the form data and saves the file, and parameterises the fields into JSON
const formidable = require('formidable')</code></pre>

&nbsp;

We&#39;ll begin by building the website for our application.&nbsp; We&#39;ll run it using Express with Pug as the page templating&nbsp;tool.&nbsp; The other tools are for handling and creating the forms on the server and sending them to Mindee.

&nbsp;

<h2>Private Token</h2>

&nbsp;

We&#39;ll use environmental variables to store the Mindee API Token:

&nbsp;

<pre>
<code class="language-javascript">let receiptToken = process.env.receiptToken;</code></pre>

&nbsp;

This keeps the token private and secure.&nbsp; In Glitch, if you remix this code, you&#39;ll have to add your token to the .env file for the API call to work correctly.&nbsp;&nbsp;

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/09/18/screenshot-2020-09-18-at-182558.png" />

&nbsp;

To create your own private token, visit the <a href="https://platform.mindee.net/">Mindee dashboard</a>, create an account, confirm your email, and enter the &quot;Receipt Parsing&quot; section of the dashboard.&nbsp; Click the &quot;Credentials&quot; tab to create your token:

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/09/03/new_token.jpg" />

&nbsp;

Add this value to the .env file.

&nbsp;

<h2>Webpage</h2>

&nbsp;

The pages we are buiiding here have no CSS, but we&#39;d store that file in the public folder.&nbsp; Just in case you are more ambitious, we&#39;ll open up the public folder available to the browser, and make sure that Pug is the tool used for creating the pages.

&nbsp;

<pre>
<code class="language-javascript">// make all the files in 'public' available
// https://expressjs.com/en/starter/static-files.html
app.use(express.static("public"));
//set up pug as view engine
app.set('view engine','pug');
// https://expressjs.com/en/starter/basic-routing.html
</code></pre>

&nbsp;

&nbsp;The first page is rendered when an initial GET request is made:

&nbsp;

<pre>
<code class="language-javascript">app.get("/", (request, response) =&gt; {
 return response.render('index');
});
</code></pre>

&nbsp;

The index.pug file is a simple page with a form.&nbsp; Users are invited to add a receipt image, and click the upload button. The source is really basic:

&nbsp;

<pre>
<code>head
 link(rel='stylesheet', href='../style.css')
 link(rel='icon', href='icon.ico', type='image/x-icon')
body
 header.header 
 h1 Node.js Receipt parsing

 h2 please upload an image of your receipt

 
 form(action="/" method = "POST" enctype="multipart/form-data").form 
 p
 |Your reciept
 input(type='file', name='imageSource').input 
 input(type='submit', value='Submit')</code></pre>

&nbsp;

This renders the following in your browser:

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/09/18/screenshot-2020-09-18-at-183155.png" />

&nbsp;

When the user adds their receipt and presses submit, the image is sent up to&nbsp;the server:

&nbsp;

<h2>Receipt image management</h2>

&nbsp;

The form sends the image to the server root via POST.&nbsp; When it comes in, formidable reads the form and parses out the files.&nbsp; These files are saved in a temporary directory (with a temporary name). For simplicity, I rename the temp file with the original receipt filename, and send this off to the makeRequest function (that will send the image to Mindee for parsing:

&nbsp;

<pre>
<code class="language-javascript">app.post("/", (request, response) =&gt; {

	//this sends in a form with an image
 //formidable reads the form, saves the image
	let form = new formidable.IncomingForm({maxFileSize : 2000 * 1024 }); //2 MB
	
	form.parse(request, (err, fields, files) =&gt; {
 if (err) {
		 console.error('Error', err);
		 throw err;
 }
	//PARSED FORM
 	console.log("files data", JSON.stringify(files.imageSource));
 let imageName = path.parse(files.imageSource.name).name;
 let imagePath = files.imageSource.path;
 let imageType = files.imageSource.type;
	let imageSize = files.imageSource.size;
	
	//FORMIDIABLE USES A RANDOM NAME ON UPLOAD. RENAME
	let newImagePath = imagePath+ imageName;
 fs.rename(imagePath, newImagePath, function (err) {
 	 if (err) throw err;
 		console.log('File uploaded and moved!');
		//FILE IS RENAMED
		//NOW UPLOAD IT TO MINDEE WITH THE MAKEREQUEST FUNCTION
		makeRequest(newImagePath);
 });
</code></pre>

&nbsp;

&nbsp;

<h2>Receipt extraction</h2>

&nbsp;

The makeRequest function takes the image and sends it to Mindee, receives the response and sends the&nbsp;relevent fields back to the user. The axios library sends the data to Mindee, and we wait for the results:

First we take the image, and place it into a few form called &#39;data; with key &quot;file&quot;.&nbsp;&nbsp;Then we create a request configuration pointing to the Mindee endpoint, insterting our token and the &#39;data&#39; form with the image

&nbsp;

<pre>
<code class="language-javascript">	async function makeRequest(newImagePath) {
 	let data = new FormData()
 		data.append('file', fs.createReadStream(newImagePath))
 		const config = {
 			 method: 'POST',
 			 url: 'https://api.mindee.net/products/expense_receipts/v2/predict',
 			 headers: { 
 				 'X-Inferuser-Token':receiptToken,
 			 ...data.getHeaders()
 				 },
				data
 		}
		console.log("config" ,config);
		try {
			let apiResponse = await axios(config)
			console.log(" api response", apiResponse.data);
		 //pull out the data I want to show on the page
			let predict = apiResponse.data.predictions;
			let merchant = apiResponse.data.predictions[0].merchant.name;
			let merchantType = apiResponse.data.predictions[0].category.value;
			let tax = apiResponse.data.predictions[0].taxes.amount;
			let total = apiResponse.data.predictions[0].total.amount

			console.log (merchant, merchantType, tax, total);
			return response.render('receipt',{merchant, tax, merchantType, total});
				 
		} catch (error) {
 			console.log(error)
 			}

 }
 
 });</code></pre>

&nbsp;

&nbsp;

<h2>Parsing the results</h2>

&nbsp;

The response comes back as a JSON file of data (for full details, <a href="https://mindee.com/documentation/apis/tutorial-parsing-receipts-with-mindees-api">read more here</a>).&nbsp;&nbsp;In&nbsp;this case, we will just extract 4 pieces of data: Merchant, Merchant Type, Taxes and Total.

&nbsp;

I extract these from the API&nbsp;response into&nbsp;variables, and send them&nbsp;to receipt.pug for rendering to the end user.

&nbsp;

<h2>Showing the results</h2>

&nbsp;

The receipt.pug file is again extremely basic, rendering just the four variables on the page, and a link back to the original form.

&nbsp;

<pre>
<code>body
 header.header 
 
 
 
 p here's the data extracted:
	
 p Merchant: #{merchant}
 p Merchant Type: #{merchantType}
 p Tax: #{tax}
 p Total #{total}
 a(href='/') Parse another receipt!</code></pre>

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/09/20/screenshot-2020-09-20-at-205753.png" />

&nbsp;

If course, you can remix this <a href="https://glitch.com/~grove-tree-sprite">Glitch</a> and change the output to better suit your needs.

&nbsp;

<h2>Conclusion</h2>

In this post, we&#39;ve walked through the steps to create a NodeJS application that receives a receipt, uploads it to Mindee, extracts the results, and presents them back to the user.&nbsp; Give the code a try, and let us know what you think through the chat box at the bottom of the page.

&nbsp;

&nbsp;

You might just want to integrate Mindee's API into your existing NodeJS Service.  In this post, we'll walk through the steps to create a receipt parsing engine running with Mindee's Receipt API and Node.

<h1>Getting started</h1>

<h2>&nbsp;</h2>

&nbsp;

Get started with receipt parsing in node.js.

&nbsp;

<h2>1. Install the mindee node.js helper library</h2>

&nbsp;

The easiest way to install&nbsp;the mindee node.js helper&nbsp;is from&nbsp;<a href="https://npmjs.org/">NPM</a>. You can run the command below from your project directory to install the library:

&nbsp;

<pre>
<code class="language-bash">npm install mindee</code></pre>

&nbsp;

&nbsp;

<h2>2. Instantiate your Client</h2>

&nbsp;

The&nbsp;<code>mindee.Client</code>&nbsp;needs your API credentials. You can either pass these directly to the constructor (see the code below) or via environment variables.

&nbsp;

<pre>
<code class="language-javascript">const { Client } = require("mindee");

const mindeeClient = new Client({
 receiptToken: "yourReceiptToken",
});</code></pre>

&nbsp;

The <code>mindee.Client</code> can take multiple parameters. Some of those parameters can also be set&nbsp;using environment&nbsp;variables. For any parameter, if the env variable and the parameter are both defined, the parameter used will be the one set in the <code>mindee.Client</code>. This is the list of the different parameters:

&nbsp;

<ul>
	<li>receiptToken (env variable :&nbsp;<code>MINDEE_RECEIPT_TOKEN</code>) The invoice API token</li>
	<li>throwOnError (<code>true</code>&nbsp;by default) If&nbsp;<code>true</code>, the SDK will throw an error when the API response status is different from 200</li>
	<li>debug (env variable :&nbsp;<code>MINDEE_DEBUG</code>,&nbsp;<code>false</code>&nbsp;by default) If&nbsp;<code>true</code>, logging will be in debug mode</li>
</ul>

&nbsp;

&nbsp;

We suggest storing your credentials as environment variables. Why? You&#39;ll never have to worry about committing your credentials and accidentally posting them somewhere public.

&nbsp;

&nbsp;

&nbsp;

<h2>3. Parse a receipt</h2>

&nbsp;

Using the parse method from&nbsp;<code>Client.receipt</code>, you can pass any image or pdf file to get the receipt data.

&nbsp;

<pre>
<code class="language-javascript">const { Client } = require("mindee");

const mindeeClient = new Client({
 receiptToken: "yourReceiptToken"
});

mindeeClient.receipt.parse(
 "path/to/file.jpg",
 'path',
 version = "3",
 cutPdf = true,
 includeWords = false
)
 .then((res) =&gt; {
 console.log(res.receipt);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

&nbsp;

<h3>Input types</h3>

&nbsp;

You can pass your input file in three ways:

&nbsp;

From file path

<pre>
<code class="language-javascript">mindeeClient.receipt.parse(
 "path/to/file.jpg",
 'path',
)
 .then((res) =&gt; {
 console.log(res.receipt);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

&nbsp;From a file object

<pre>
<code class="language-javascript">const fs = require("fs");

const file = fs.createReadStream("./path/to/img/or/pdf");

mindeeClient.receipt.parse(
 file,
 'stream',
)
 .then((res) =&gt; {
 console.log(res.receipt);
 })
 .catch((err) =&gt; {
 console.error(err);
 });
</code></pre>

&nbsp;

From a base64

<pre>
<code class="language-python">const base64 = fs.readFileSync("./inv.pdf", {
 encoding: "base64",
});

mindeeClient.receipt.parse(
 file,
 'base64',
)
 .then((res) =&gt; {
 console.log(res.receipt);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

&nbsp;

Result data&nbsp;structure

&nbsp;

The <code>result</code>&nbsp;object returned by the&nbsp;parse method contains the following elements:

&nbsp;

<pre>
<code class="language-javascript">mindeeClient.receipt.parse(
 '/path/to/file,
 'path',
)
 .then((result) =&gt; {
 console.log(res.receipt);
 console.log(res.receipts);
 console.log(res.httpResponse);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

result.receipt

&nbsp;

The receipt attribute&nbsp;is the Receipt object constructed by gathering all the pages into a single document setting the fields by checking the maximum confidence score of all pages.

&nbsp;

result.receipts

&nbsp;

For multi-pages pdf, the receipts&nbsp;attribute is a list of receipt objects, each object is constructed using a unique page of the pdf

&nbsp;

result.http_response

&nbsp;

Contains the full Mindee API HTTP response object

&nbsp;

&nbsp;

More details about the extracted fields in&nbsp;<a href="https://mindee.com/documentation/apis/tutorial-parsing-receipts-with-mindees-api/parse-receipts-nodejs/receipt-fields">the Receipt fields section</a>

&nbsp;

&nbsp;

Get started with receipt parsing using Mindee API and node.js

<h1>Receipt fields</h1>

&nbsp;

&nbsp;

<h2>General fields structure</h2>

&nbsp;

Each receipt object contains a set of different fields.

&nbsp;

To access a receipt object, you need to create a&nbsp;<code>mindee.Client&nbsp;</code>and call the&nbsp;<code>Client.receipt.parse</code>&nbsp;method:

&nbsp;

<pre>
<code>const { Client } = require("mindee");

const mindeeClient = new Client({
 receiptToken: "yourReceiptToken"
});

mindeeClient.receipt.parse(
 "path/to/file.jpg",
 'path',
 version = "3",
 cutPdf = true,
 includeWords = false
)
 .then((res) =&gt; {
 console.log(res.receipt);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

Whatever the field type, each of them contains the four following attributes:

&nbsp;

<ul>
	<li>Value: (Str or Float depending on the field type), corresponds to the field value. Set to None if the field was not extracted. For&nbsp;reconstructed fields, the value attribute might not be present in the field (this will be changed in a later version of the node.js SDK)</li>
	<li>Probability: (Float), the confidence score of the field prediction.</li>
	<li>bbox:(Array[Float]), contains the relative vertices coordinates of the bounding box containing the field in the image. If the field is not written, the bbox is an empty array.&nbsp;</li>
	<li>reconstructed: (Bool), True if the field was reconstructed using other fields.</li>
	<li>pageNumber: (int), the page number of the extracted field</li>
</ul>

&nbsp;

Depending on the Field type, there can be extra attributes.

&nbsp;

&nbsp;

&nbsp;

<h2>Total amounts</h2>

&nbsp;

receipt.totalIncl:&nbsp;Total amount including taxes

receipt.totalExcl:&nbsp;Total amount excluding taxes

receipt.totalTax:&nbsp;Total tax value reconstructed from tax lines

&nbsp;

<pre>
<code class="language-javascript">
mindeeClient.receipt.parse(
 "path/to/file.jpg",
 'path'
)
 .then((res) =&gt; {
 console.log(res.receipt.totalIncl);
 console.log(res.receipt.totalExcl);
 console.log(res.receipt.totalTax);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

For a single receipt, this code print in your console something like:

&nbsp;

<pre>
<code class="language-json">{
 pageNumber: 0,
 reconstructed: false,
 value: 10.2,
 probability: 1,
 bbox: [
 [ 0.549, 0.619 ],
 [ 0.715, 0.619 ],
 [ 0.715, 0.64 ],
 [ 0.549, 0.64 ]
 ]
}
{
 pageNumber: 0,
 reconstructed: true,
 value: 8.5,
 probability: 1,
 bbox: []
}
{
 pageNumber: 0,
 reconstructed: true,
 value: 1.7,
 probability: 1,
 bbox: []
}
</code></pre>

&nbsp;

<h2>Taxes</h2>

&nbsp;

receipt.taxes:&nbsp;Array of Tax fields

Each of the tax fields has two extra attributes:

&nbsp;

<ul>
	<li>code: (String), Optional tax code. (HST, GST... for Canadian; City Tax, State tax for US, etc..)</li>
	<li>rate: (Float), Optional tax rate.</li>
</ul>

&nbsp;

<pre>
<code class="language-javascript">mindeeClient.receipt.parse(
 "path/to/file.jpg",
 'path'
)
 .then((res) =&gt; {
 console.log(res.receipt.taxes);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

<pre>
<code class="language-json">[
 {
 pageNumber: 0,
 reconstructed: false,
 value: 1.7,
 probability: 1,
 bbox: [ [Array], [Array], [Array], [Array] ],
 rate: 20
 }
]
</code></pre>

&nbsp;

&nbsp;

&nbsp;

<h2>Dates</h2>

&nbsp;

Each date field comes with an extra attribute:

&nbsp;

<ul>
	<li>dateObject:: (Datetime), DateTime object&nbsp;</li>
</ul>

&nbsp;

&nbsp;

&nbsp;receipt.date: Payment date of the receipt

&nbsp;

<pre>
<code class="language-javascript">mindeeClient.receipt.parse(
 "path/to/file.jpg",
 'path'
)
 .then((res) =&gt; {
 console.log(res.receipt.taxes);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

<pre>
<code class="language-json">{
 pageNumber: 0,
 reconstructed: false,
 value: '2016-02-26',
 probability: 0.99,
 bbox: [
 [ 0.479, 0.173 ],
 [ 0.613, 0.173 ],
 [ 0.613, 0.197 ],
 [ 0.479, 0.197 ]
 ],
 dateObject: '2016-02-26T00:00:00.000Z'
}
</code></pre>

&nbsp;

&nbsp;

<h2>Supplier information</h2>

&nbsp;

&nbsp;receipt.supplier:&nbsp;Supplier name as written in the receipt&nbsp;(logo)

&nbsp;

<pre>
<code class="language-javascript">mindeeClient.receipt.parse(
 "path/to/file.jpg",
 'path'
)
 .then((res) =&gt; {
 console.log(res.receipt.merchantName);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

<pre>
<code class="language-json">{
 pageNumber: 0,
 reconstructed: false,
 value: 'CLACHAN',
 probability: 0.71,
 bbox: [
 [ 0.394, 0.068 ],
 [ 0.477, 0.068 ],
 [ 0.477, 0.087 ],
 [ 0.394, 0.087 ]
 ]
}
</code></pre>

&nbsp;

&nbsp;

<h2>Locale and currency</h2>

&nbsp;

locale: Language ISO code

Contains an extra attribute:

&nbsp;

<ul>
	<li>currency: (String), ISO currency code</li>
</ul>

&nbsp;

<pre>
<code class="language-javascript">mindeeClient.receipt.parse(
 "path/to/file.jpg",
 'path'
)
 .then((res) =&gt; {
 console.log(res.receipt.locale);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

<pre>
<code class="language-json">{
 pageNumber: 0,
 reconstructed: false,
 value: 'en-GB',
 probability: 0.82,
 bbox: [],
 language: 'en',
 country: 'GB',
 currency: 'GBP'
}
</code></pre>

&nbsp;

&nbsp;

<h2>&nbsp;Category</h2>

&nbsp;

category: Receipt category among the list:&nbsp; toll, food, parking, transport, accommodation, gasoline, miscellaneous

&nbsp;

<pre>
<code class="language-javascript">mindeeClient.receipt.parse(
 "path/to/file.jpg",
 'path'
)
 .then((res) =&gt; {
 console.log(res.receipt.category);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

<pre>
<code class="language-json">{
 pageNumber: 0,
 reconstructed: false,
 value: 'food',
 probability: 0.99,
 bbox: []
}
</code></pre>

&nbsp;

&nbsp;

<h2>&nbsp;Time</h2>

&nbsp;

time: Time of purchase with 24 hours formatting (hh:mm)

&nbsp;

<pre>
<code class="language-javascript">mindeeClient.receipt.parse(
 "path/to/file.jpg",
 'path'
)
 .then((res) =&gt; {
 console.log(res.receipt.time);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

<pre>
<code class="language-json">{
 pageNumber: 0,
 reconstructed: false,
 value: '15:20',
 probability: 0.99,
 bbox: [
 [ 0.62, 0.173 ],
 [ 0.681, 0.173 ],
 [ 0.681, 0.191 ],
 [ 0.62, 0.191 ]
 ]
}
</code></pre>

&nbsp;

&nbsp;

Access receipt fields using Mindee API and node.js

node.js

Parsing Receipts

<h1>Parsing Invoices with Mindee&rsquo;s API</h1>

&nbsp;

&nbsp;

Scanning and inputting data from invoices into accounting systems is a slow and tedious process.&nbsp; Using Mindee&rsquo;s invoice parsing API, all of the pertinent data can be automatically and accurately extracted from your invoice in seconds, allowing for fast and painless data entry.&nbsp;&nbsp;

&nbsp;

Interested in learning more? In this tutorial, we will walk through the steps to use Mindee&rsquo;s Invoice API.&nbsp;

&nbsp;

<h2>API Prerequisites</h2>

&nbsp;

<ol>
	<li style="list-style-type:decimal">You&rsquo;ll need Mindee account.<a href="http://platform.mindee.net/signup"> Sign up</a><a href="http://platform.mindee.net/signup">&nbsp;for free</a>.&nbsp; Then confirm your email to login.</li>
	<li style="list-style-type:decimal">An invoice.&nbsp; Use a recently received invoice, or do a <a href="https://www.google.com/search?q=invoice">Google Image search</a> for a invoice&nbsp;and download a few to test with.&nbsp;&nbsp;</li>
</ol>

&nbsp;

<h2>Setting up the API</h2>

&nbsp;

Log into you Mindee account and access your Invoices API environment by clicking the Invoices card :

&nbsp;

<h1><img alt="" src="https://mindee-public-website.s3.amazonaws.com/blog/2020/09/11/screenshot-2020-09-11-at-192210.png" style="height:558px; width:400px" /></h1>

To activate the API, click the &ldquo;Try for Free&rdquo; button to access the free tier. You&rsquo;ll land on the dashboard page - this page will show the usage of your account with the Invoice API - of course, it is empty now.&nbsp; On the left navigation, there are links to &ldquo;Documentation&rdquo;, &ldquo;Credentials&rdquo; and &ldquo;Live Interface&rdquo;.&nbsp; The docs tab has all of the technical details you&rsquo;ll need to build for the Invoce API endpoint, and the Live Interface is a cool interactive demo. Rather than try out the demo, we want to build with the API,&nbsp; so click on &ldquo;Credentials&rdquo; to create an API token.

&nbsp;

Add a new token. In this example, I&rsquo;ve named it &ldquo;Tutorial&rdquo;

<h1> 
<img alt="" src="https://mindee-public-website.s3.amazonaws.com/blog/2020/09/01/pasted-image-0-1.png" style="height:247px; width:500px" /></h1>

&nbsp;

Click &ldquo;Add New Key&rdquo; and you&rsquo;ll be able to see your API token.

Now, we are ready to make an API call.&nbsp; In this example, we&rsquo;ll be using cURL.

&nbsp;

<pre>
<code class="language-bash">curl -X POST \ https://api.mindee.net/products/invoices/v1/predict 
 -H 'X-Inferuser-Token: {apiToken}' 
 -H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' 
 -F file=@/path/to/your/file.png</code></pre>

&nbsp;

&nbsp;

Simply replace ```{apiToken}``` with your new API token and ```/path/to/your/file/png``` with the path to your invoice - wither an image or a pdf.&nbsp;

&nbsp;

NOTE: You can also copy this code right from the documentation tab of the API with your API token inserted for you.

Here&rsquo;s the invoice I used - the roofer who fixed the gasket around our chimney this spring (I&rsquo;ve obfuscated my home address in the image):

&nbsp;

<h1><img alt="" src="https://mindee-public-website.s3.amazonaws.com/blog/2020/09/11/screenshot-2020-09-11-at-193913.png" style="height:383px; width:500px" /></h1>

&nbsp;

Pasting the cURL sample into my terminal, I hit enter and about a second later, I received a JSON response with the receipt details.&nbsp; The full JSON can be accessed here (link to JSON file).&nbsp; Since the response is quite verbose, we will walk through the various fields section by section.

&nbsp;

<h2>Extracted fields</h2>

&nbsp;

<h2>Summary &amp; Documents section</h2>

&nbsp;

The first two sections of the response contain information about the API call made:

<pre>
<code class="language-json">	"call": {
		"endpoint": { "name": "invoices", "version": "1.0" },
		"finished_at": "2020-09-11T18:39:58+00:00",
		"id": "3b4ac75a-f800-466b-bfc4-56c8f8e75134",
		"n_documents": 1,
		"n_inputs": 1,
		"processing_time": 1.047,
		"started_at": "2020-09-11T18:39:57+00:00"
	},
	"documents": [
		{
			"id": "5d72b0c2-16dd-481d-9e8b-031c5d1ab33a",
			"name": "Screenshot 2020-09-11 at 19.39.13.png"
		}
	],</code></pre>

&nbsp;

The call section tells us that we ran on the v1 of the invoice endpoint, uploading one document that is one page long.&nbsp; After a second, the file was processed, and the response transmitted back to me.&nbsp; The documents section gives the Mindee id for the file, and the filename (and you can tell I used a screenshot of the original invoice).

&nbsp;

<h2>Predictions:&nbsp;</h2>

&nbsp;

This is the fun part, where the API extraxcts pertinent details form teh invoice automatcilly, saving tedious manual data entry.&nbsp; Every invoice is different, and will have different fields. This invoice, being sent from the UNited States will not have some common European features like and IBAN or Tax Id.&nbsp;&nbsp;

Other peices are rpesent in this invoice, but not picked up by the API.&nbsp; We are planning multiple additional releases to this API in 2020 that will continue to improve the accuracy and precision of the results.&nbsp;&nbsp;

&nbsp;

<h3>Company Number</h3>

In this case, there is no company number in the invoice:

<pre>
<code class="language-json">"company_number": {
				"probability": 0,
				"segmentation": { "bounding_box": [] },
				"type": "N/A",
				"value": "N/A"
			},</code></pre>

<h2>&nbsp;</h2>

<h3>Due Date</h3>

My invoice does have a due date, but it was not extracted.&nbsp;

<pre>
<code class="language-json">			"due_date": {
				"iso": "N/A",
				"probability": 0,
				"raw": "N/A",
				"segmentation": { "bounding_box": [] }
			},</code></pre>

<h2>&nbsp;</h2>

<h3>Invoice Date</h3>

The work was done in late June of this year, and the API correctly extracts the invoice date, along with four [x,y] points boxing in the value:

<pre>
<code class="language-json">			"invoice_date": {
				"iso": "2020-06-29",
				"probability": 0.99,
				"segmentation": {
					"bounding_box": [
						[ 0.766, 0.392 ],
						[ 0.855, 0.392 ],
						[ 0.855, 0.416 ],
						[ 0.766, 0.416 ]
					]
				}
			},</code></pre>

<h3>&nbsp;</h3>

<h3>Invoice Number</h3>

The invoice is numbered 1277, which is extracted by the API, along with the four [x,y] corrdinates denoting the location in the image.

<pre>
<code class="language-json">			"invoice_number": {
				"probability": 0.99,
				"segmentation": {
					"bounding_box": [
						[ 0.766, 0.363 ],
						[ 0.807, 0.363 ],
						[ 0.807, 0.382 ],
						[ 0.766, 0.382 ]
					]
				},
				"value": "1277"
			},</code></pre>

<h3>&nbsp;</h3>

<h3>Locale</h3>

Based on the information on the invoice, the API is able to predict (with 82% confidence) that this bill originated in the United States, is in USD, and is in English.

<pre>
<code class="language-json">"locale": { "currency": "USD", "language": "en", "probability": 0.82 },</code></pre>

<h2>&nbsp;</h2>

<h3>Orientation</h3>

The invoice did not require rotation to be parsed.

<pre>
<code class="language-json">"orientation": { "degrees": 0, "probability": 0.99 },</code></pre>

<h2>&nbsp;</h2>

<h3>Payment Details</h3>

The invoice does not contain any payment details, so were not extracted.

<pre>
<code class="language-json">"payment_details": {
				"iban": "N/A",
				"probability": 0,
				"segmentation": { "bounding_box": [] }
			},</code></pre>

<h3>&nbsp;</h3>

<h3>Supplier</h3>

The API is only 50% confident in this result, but it does extract the name of the company (despite without any spaces in the name).&nbsp;&nbsp;

<pre>
<code class="language-json">			"supplier": {
				"probability": 0.5,
				"segmentation": {
					"bounding_box": [
						[ 0.446, 0.066 ],
						[ 0.561, 0.066 ],
						[ 0.561, 0.083 ],
						[ 0.446, 0.083 ]
					]
				},
				"value": "SILVERHAMMER"
			},</code></pre>

<h3>&nbsp;</h3>

<h3>Tax Id</h3>

Companies in the US generally do not list their Tax IDs on invoices, so this was not found.

<pre>
<code class="language-json">			"tax_id": {
				"probability": 0,
				"segmentation": { "bounding_box": [] },
				"value": "N/A"
			},</code></pre>

<h3>&nbsp;</h3>

<h3>Taxes</h3>

The API did not extract the texes, or the percentage tax rate (we have an update later this year t better extract US tax details.)

<pre>
<code class="language-json">"taxes": [],</code></pre>

<h2>&nbsp;</h2>

<h3>Total without Taxes</h3>

The API did not extract the taxes, but did get the cost before taxes with 99% confidence.&nbsp;&nbsp;

<pre>
<code class="language-json">	"total_excl": {
				"amount": 350,
				"probability": 0.99,
				"segmentation": {
					"bounding_box": [
						[ 0.874, 0.803 ],
						[ 0.931, 0.803 ],
						[ 0.931, 0.825 ],
						[ 0.874, 0.825 ]
					]
				}
			},</code></pre>

<h2>&nbsp;</h2>

<h3>Total with Taxes</h3>

The total with taxes was alos extracted accurately with 99% confidence, meaning that the tax value could easily be calculated with a simple difference.

<pre>
<code class="language-json">"total_incl": {
				"amount": 380.45,
				"probability": 0.99,
				"segmentation": {
					"bounding_box": [
						[ 0.842, 0.897 ],
						[ 0.933, 0.897 ],
						[ 0.933, 0.93 ],
						[ 0.842, 0.93 ]
					]
				}
			}</code></pre>

&nbsp;

<h2>Conclusion</h2>

Manual entry of invoices is a time consuming task. Adding the Mindee API to your system will simplify the extraction and make your accounts paybale processes run more accurately and smoothly.&nbsp; Do you have more questions? Click the chat button at the bottom right.

&nbsp;

<h1>&nbsp;</h1>

&nbsp;

&nbsp;

Inputting data from invoices into accounting systems is a slow and tedious process.  Using Mindee’s invoice parsing API, extract data from invoices automatically and accurately in seconds, allowing for fast and painless data entry.

<h1>Invoice OCR using node.js</h1>

&nbsp;

To get started with invoice parsing using Mindee API in node.js, you can use our node helper library;

&nbsp;

<ul>
	<li>1.&nbsp;<a href="https://mindee.com/documentation/apis/invoice-parsing/invoice-parse-nodejs/getting-started">Node.js SDK overview</a></li>
	<li>2.&nbsp;<a href="https://mindee.com/documentation/apis/invoice-parsing/invoice-parse-nodejs/invoice-fields">Extracted invoice fields</a></li>
</ul>

&nbsp;

If you want to see how to use the API using HTTP requests, you can follow this guide.

&nbsp;

&nbsp;

&nbsp;

<h2>Tutorial</h2>

&nbsp;

In&nbsp;this example, we will build from scratch a NodeJS based web application that accepts an&nbsp;image or PDF of an invoice, and return&nbsp;several points of data extracted from the invoice.

&nbsp;

The code for the sample code is available on <a href="https://github.com/dougsillars/mindee_parse_invoice">Github</a>, and also available on <a href="https://glitch.com/edit/#!/recondite-icy-wanderer?path=server.js%3A13%3A4">Glitch</a>.&nbsp; For simplicity, we&#39;ll follow along in the Glitch example.

&nbsp;

<div class="youtube-embed-wrapper" style="height:0; overflow:hidden; padding-bottom:56.25%; padding-top:30px; position:relative"><iframe frameborder="0" height="360" src="https://www.youtube.com/embed/0lV_PtELbqk" style="position:absolute;top:0;left:0;width:100%;height:100%" width="640"></iframe></div>

&nbsp;

<h2>Glitch example</h2>

&nbsp;

In the server.js, we need to initialise all of our required modules:

&nbsp;

<pre>
<code class="language-javascript">const express = require('express');
const app = express();
const pug = require('pug');

const path = require('path');
const fs = require("fs");
const axios = require('axios');
let FormData = require('form-data');
//formidable takes the form data and saves the file, and parameterises the fields into JSON
const formidable = require('formidable')
</code></pre>

&nbsp;

We&#39;ll begin by building the website for our application.&nbsp; We&#39;ll run it using Express with Pug as the page templating&nbsp;tool.&nbsp; The other tools are for handling and creating the forms on the server and sending them to Mindee.

&nbsp;

<h2>Private Token</h2>

&nbsp;

We&#39;ll use environmental variables to store the Mindee API Token:

&nbsp;

<pre>
<code class="language-javascript">let invoiceToken = process.env.invoiceToken;</code></pre>

&nbsp;

This keeps the token private and secure.&nbsp; In Glitch, if you remix this code, you&#39;ll have to add your token to the .env file for the API call to work correctly.&nbsp;&nbsp;

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/09/24/screenshot-2020-09-24-at-121954.png" style="height:158px; width:400px" />

&nbsp;

To create your own private token, visit the <a href="https://platform.mindee.net/">Mindee dashboard</a>, create an account, confirm your email, and enter the &quot;Invoice&nbsp;API&quot; section of the dashboard.&nbsp; Click the &quot;Credentials&quot; tab to create your token:

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/09/03/new_token.jpg" />

&nbsp;

Add this value to the .env file.

&nbsp;

<h2>Webpage</h2>

&nbsp;

The pages we are buiiding here have no CSS, but we&#39;d store that file in the public folder.&nbsp; Just in case you are more ambitious, we&#39;ll open up the public folder available to the browser, and make sure that Pug is the tool used for creating the pages.

&nbsp;

<pre>
<code class="language-javascript">// make all the files in 'public' available
// https://expressjs.com/en/starter/static-files.html
app.use(express.static("public"));
//set up pug as view engine
app.set('view engine','pug');
// https://expressjs.com/en/starter/basic-routing.html
</code></pre>

&nbsp;

&nbsp;The first page is rendered when an initial GET request is made:

&nbsp;

<pre>
<code class="language-javascript">app.get("/", (request, response) =&gt; {
 return response.render('index');
});
</code></pre>

&nbsp;

The index.pug file is a simple page with a form, where users can add an invoice&nbsp;image, and click the upload button. Not a lot to see here:

&nbsp;

<pre>
<code class="language-html">head
 link(rel='stylesheet', href='../style.css')
 link(rel='icon', href='icon.ico', type='image/x-icon')
body
 header.header 
 
 
 h1 Node.js Invoice extraction

 h2 please upload an image or PDF of your invoice

 
 form(action="/" method = "POST" enctype="multipart/form-data").form 
 p
 |Your reciept
 input(type='file', name='imageSource').input 
 input(type='submit', value='Submit')</code></pre>

&nbsp;

This renders the following in your browser:

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/09/24/screenshot-2020-09-24-at-122314.png" style="height:290px; width:300px" />

&nbsp;

When the user adds an invoice and presses submit, the image is sent up to the server for processesing

&nbsp;

<h2>Invoice image processing on the server</h2>

&nbsp;

The form sends the image to the server root via POST.&nbsp; When it comes in, formidable reads the form and parses out the files.&nbsp; These files are saved in a temporary directory (with a temporary name). For simplicity, I rename the temp file with the original invoice filename.&nbsp; I do this as the filename in the API response will match whatever is uploaded to Mindee, so keeping the filename the same makes identification&nbsp;easier.&nbsp;The file is then sent to the&nbsp;makeRequest function:

&nbsp;

<pre>
<code class="language-javascript">app.post("/", (request, response) =&gt; {

	//this sends in a form with an image
 //formidable reads the form, saves the image
	let form = new formidable.IncomingForm({maxFileSize : 2000 * 1024 }); //2 MB
	
	form.parse(request, (err, fields, files) =&gt; {
 if (err) {
		 console.error('Error', err);
		 throw err;
 }
	//PARSED FORM
 	console.log("files data", JSON.stringify(files.imageSource));
 let imageName = path.parse(files.imageSource.name).name;
 let imagePath = files.imageSource.path;
 let imageType = files.imageSource.type;
	let imageSize = files.imageSource.size;
	
	//FORMIDIABLE USES A RANDOM NAME ON UPLOAD. RENAME
	let newImagePath = imagePath+ imageName;
 fs.rename(imagePath, newImagePath, function (err) {
 	 	if (err) throw err;
 		 console.log('File uploaded and moved!');
		 //FILE IS RENAMED
		 //NOW UPLOAD IT TO MINDEE WITH THE MAKEREQUEST FUNCTION
		 makeRequest(newImagePath);
 	});
	</code></pre>

&nbsp;

<h2>Invoice data extraction</h2>

&nbsp;

The makeRequest function takes the image and sends it to Mindee, receives the response and sends the&nbsp;relevent fields back to the user. The axios library sends the data to Mindee, and we wait for the results:

First we take the image, and place it into a few form called &#39;data; with key &quot;file&quot;.&nbsp;&nbsp;Then we create a request configuration pointing to the Mindee endpoint, insterting our token and the &#39;data&#39; form with the image

&nbsp;

<pre>
<code class="language-javascript">		async function makeRequest(newImagePath) {
 		 let data = new FormData()
 		 data.append('file', fs.createReadStream(newImagePath))
	
		//	console.log("form data ", data);
	
 		 const config = {
 			 method: 'POST',
 			 url: 'https://api.mindee.net/products/invoices/v1/predict',
 			 headers: { 
 				 'X-Inferuser-Token':invoiceToken,
 				 ...data.getHeaders()
 				 },
				 data
 			 }
			 console.log("config" ,config);
			 try {
				 let apiResponse = await axios(config)
				 console.log(" api response", apiResponse.data);
				 
			</code></pre>

&nbsp;

&nbsp;

<h2>Results</h2>

&nbsp;

The response comes back as a JSON file of data that has been extracted from the invoice (for full details, [read more here](https://mindee.com/documentation/apis/invoice-parsing)).&nbsp;&nbsp;We&#39;ll extract the invoice number, date, company name, the currency, and the total before and after taxes.

&nbsp;

I extract these from the APII response into&nbsp;variables, and send them&nbsp;to invoice.pug for rendering to the end user.

&nbsp;

<pre>
<code class="language-javascript"> //pull out the data I want to show on the page
let currency = apiResponse.data.predictions[0].locale.currency;
let invoice = apiResponse.data.predictions[0].invoice_number.value;
let merchant = apiResponse.data.predictions[0].supplier.value;
let date = apiResponse.data.predictions[0].invoice_date.iso;
let beforeTax = apiResponse.data.predictions[0].total_excl.amount;
let total = apiResponse.data.predictions[0].total_incl.amount
				 
console.log (invoice, currency, merchant, date, beforeTax, total);
return response.render('invoice',{invoice, currency, merchant, date, beforeTax, total});
				</code></pre>

&nbsp;

&nbsp;

<h2>Displaying the data</h2>

&nbsp;

The invoice.pug file renders&nbsp;just the six&nbsp;results we extracted&nbsp;and a link back to the original form.

&nbsp;

<pre>
<code>body
 header.header 
 
 
 
 p here's the data extracted:
	
 p Merchant: #{merchant}
 p invoice number: #{invoice}
 p Date: #{date}
 p Currency: #{currency}
 p Total before Taxes: #{beforeTax}
 p Total with tax #{total}
 a(href='/') Parse another invoice!</code></pre>

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/09/24/screenshot-2020-09-24-at-124750.png" style="height:327px; width:300px" />

&nbsp;

&nbsp;

If course, you can remix this <a href="https://glitch.com/~recondite-icy-wanderer">Glitch</a> and change the output to better suit your needs.

&nbsp;

&nbsp;

<h2>Conclusion</h2>

In this post, we&#39;ve walked through the steps to create a NodeJS application that receives an invoice, uploads it to Mindee, extracts the results, and presents them back to the user.&nbsp; Give the code a try, and let us know what you think through the chat box at the bottom of the page.

&nbsp;

&nbsp;

The Invoice parsing API from Mindee is a powerful tool to quickly extract crucial data from invoices. In this post, we walk though a basic NodeJS integration.

<h1>Getting started</h1>

<h2>&nbsp;</h2>

&nbsp;

Get started with Invoice parsing in node.js

&nbsp;

<h2>1. Install the mindee node.js helper library</h2>

&nbsp;

The easiest way to install&nbsp;the mindee node.js helper&nbsp;is from&nbsp;<a href="https://npmjs.org/">NPM</a>. You can run the command below from your project directory to install the library:

&nbsp;

<pre>
<code class="language-bash">npm install mindee</code></pre>

&nbsp;

&nbsp;

<h2>2. Instantiate your Client</h2>

&nbsp;

The&nbsp;<code>mindee.Client</code>&nbsp;needs your API credentials. You can either pass these directly to the constructor (see the code below) or via environment variables.

&nbsp;

<pre>
<code class="language-javascript">const { Client } = require("mindee");

const mindeeClient = new Client({
 invoiceToken: "yourInvoiceToken",
});</code></pre>

&nbsp;

The <code>mindee.Client</code> can take multiple parameters. Some of those parameters can also be set&nbsp;using environment&nbsp;variables. For any parameter, if the env variable and the parameter are both defined, the parameter used will be the one set in the <code>mindee.Client</code>. This is the list of the different parameters:

&nbsp;

<ul>
	<li>invoiceToken (env variable :&nbsp;<code>MINDEE_INVOICE_TOKEN</code>) The invoice API token</li>
	<li>throwOnError (<code>true</code>&nbsp;by default) If&nbsp;<code>true</code>, the SDK will throw an error when the API response status is different from 200</li>
	<li>debug (env variable :&nbsp;<code>MINDEE_DEBUG</code>,&nbsp;<code>false</code>&nbsp;by default) If&nbsp;<code>true</code>, logging will be in debug mode</li>
</ul>

&nbsp;

&nbsp;

We suggest storing your credentials as environment variables. Why? You&#39;ll never have to worry about committing your credentials and accidentally posting them somewhere public.

&nbsp;

&nbsp;

&nbsp;

<h2>3. Parse an invoice</h2>

&nbsp;

Using the parse method from&nbsp;<code>Client.invoice</code>, you can pass any image or pdf file to get the invoice data.

&nbsp;

<pre>
<code class="language-javascript">const { Client } = require("mindee");

const mindeeClient = new Client({
 invoiceToken: "yourInvoiceToken"
});

mindeeClient.invoice.parse(
 "path/to/file.jpg",
 'path',
 version = "2",
 cutPdf = true,
 includeWords = false
)
 .then((res) =&gt; {
 console.log(res.invoice);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

&nbsp;

<h3>Input types</h3>

&nbsp;

You can pass your input file in three ways:

&nbsp;

From file path

<pre>
<code class="language-javascript">mindeeClient.invoice.parse(
 "path/to/file.jpg",
 'path',
)
 .then((res) =&gt; {
 console.log(res.invoice);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

&nbsp;From a file object

<pre>
<code class="language-javascript">const fs = require("fs");

const file = fs.createReadStream("./path/to/img/or/pdf");

mindeeClient.invoice.parse(
 file,
 'stream',
)
 .then((res) =&gt; {
 console.log(res.invoice);
 })
 .catch((err) =&gt; {
 console.error(err);
 });
</code></pre>

&nbsp;

From a base64

<pre>
<code class="language-python">const base64 = fs.readFileSync("./inv.pdf", {
 encoding: "base64",
});

mindeeClient.invoice.parse(
 file,
 'base64',
)
 .then((res) =&gt; {
 console.log(res.invoice);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

&nbsp;

Result data&nbsp;structure

&nbsp;

The <code>result</code>&nbsp;object returned by the&nbsp;parse method contains the following elements:

&nbsp;

<pre>
<code class="language-javascript">mindeeClient.invoice.parse(
 '/path/to/file,
 'path',
)
 .then((result) =&gt; {
 console.log(res.invoice);
 console.log(res.invoices);
 console.log(res.httpResponse);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

result.invoice

&nbsp;

The invoice attribute&nbsp;is the Invoice object constructed by gathering all the pages into a single document setting the fields by checking the maximum confidence score of all pages.

&nbsp;

result.invoices

&nbsp;

For multi-pages pdf, the invoices attribute is a list of<code> Invoice</code> objects, each object is constructed using a unique page of the pdf

&nbsp;

result.http_response

&nbsp;

Contains the full Mindee API HTTP response object

&nbsp;

&nbsp;

More details about the extracted fields in&nbsp;<a href="https://mindee.com/documentation/apis/invoice-parsing/invoice-parse-nodejs/invoice-fields">the Invoice fields section</a>

&nbsp;

&nbsp;

Getting started with invoice parsing using Mindee invoice OCR and node.js

<h1>Invoice fields</h1>

&nbsp;

&nbsp;

<h2>General fields structure</h2>

&nbsp;

Each invoice object contains a set of different fields.

&nbsp;

To access an invoice object, you need to create a&nbsp;<code>mindee.Client&nbsp;</code>and call the&nbsp;<code>Client.</code>invoice<code>.parse</code>&nbsp;method:

&nbsp;

<pre>
<code>const { Client } = require("mindee");

const mindeeClient = new Client({
 invoiceToken: "yourInvoiceToken"
});

mindeeClient.invoice.parse(
 "path/to/file.jpg",
 'path',
 version = "2",
 cutPdf = true,
 includeWords = false
)
 .then((res) =&gt; {
 console.log(res.invoice);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

Whatever the field type, each of them contains the four following attributes:

&nbsp;

<ul>
	<li>Value: (Str or Float depending on the field type), corresponds to the field value. Set to None if the field was not extracted. For&nbsp;reconstructed fields, the value attribute might not be present in the field (this will be changed in a later version of the node.js SDK)</li>
	<li>Probability: (Float), the confidence score of the field prediction.</li>
	<li>bbox:(Array[Float]), contains the relative vertices coordinates of the bounding box containing the field in the image. If the field is not written, the bbox is an empty array.&nbsp;</li>
	<li>reconstructed: (Bool), True if the field was reconstructed using other fields.</li>
	<li>pageNumber: (int), the page number of the extracted field</li>
</ul>

&nbsp;

Depending on the Field type, there can be extra attributes.

&nbsp;

&nbsp;

&nbsp;

<h2>Total amounts</h2>

&nbsp;

invoice.totalIncl:&nbsp;Total amount including taxes

invoice.totalExcl:&nbsp;Total amount excluding taxes

invoice.totalTax:&nbsp;Total tax value reconstructed from tax lines

&nbsp;

<pre>
<code class="language-javascript">mindeeClient.invoice.parse(
 "invoice.png",
 'path',
)
 .then((res) =&gt; {
 console.log(res.invoice.totalIncl);
 console.log(res.invoice.totalExcl);
 console.log(res.invoice.totalTax);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

&nbsp;

<pre>
<code class="language-json">{
 pageNumber: 0,
 reconstructed: false,
 value: 38.92,
 probability: 0.99,
 bbox: [
 [ 0.865, 0.641 ],
 [ 0.955, 0.641 ],
 [ 0.955, 0.675 ],
 [ 0.865, 0.675 ]
 ]
}
{
 pageNumber: 0,
 reconstructed: false,
 value: 36.72,
 probability: 0.99,
 bbox: [
 [ 0.898, 0.709 ],
 [ 0.955, 0.709 ],
 [ 0.955, 0.734 ],
 [ 0.898, 0.734 ]
 ]
}
{
 pageNumber: 0,
 reconstructed: true,
 value: 2.2,
 probability: 0.9801,
 bbox: []
}
</code></pre>

&nbsp;

<h2>Taxes</h2>

&nbsp;

invoice.taxes:&nbsp;Array of Tax fields

Each of the tax fields has two extra attributes:

&nbsp;

<ul>
	<li>rate: (Float), Optional tax rate.</li>
</ul>

&nbsp;

<pre>
<code class="language-javascript">mindeeClient.invoice.parse(
 "invoice.png",
 'path',
)
 .then((res) =&gt; {
 console.log(res.invoice.taxes);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

<pre>
<code class="language-json">[
 {
 pageNumber: 0,
 reconstructed: false,
 value: 1.7,
 probability: 1,
 bbox: [ [Array], [Array], [Array], [Array] ],
 rate: 20
 }
]
</code></pre>

&nbsp;

&nbsp;

<h2>Dates</h2>

&nbsp;

Each date field comes with an extra attribute:

&nbsp;

<ul>
	<li>dateObject:: (Datetime), DateTime object&nbsp;</li>
</ul>

&nbsp;

&nbsp;

invoice.invoiceDate: Issuance date of the invoice

invoice.dueDate: Due date of payment of the invoice

&nbsp;

<pre>
<code class="language-javascript">mindeeClient.invoice.parse(
 "invoice.png",
 'path',
)
 .then((res) =&gt; {
 console.log(res.invoice.invoiceDate);
 console.log(res.invoice.dueDate);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

<pre>
<code class="language-json">{
 pageNumber: 0,
 reconstructed: false,
 value: '2021-01-28',
 probability: 0.99,
 bbox: [
 [ 0.395, 0.448 ],
 [ 0.474, 0.448 ],
 [ 0.474, 0.462 ],
 [ 0.395, 0.462 ]
 ],
 dateObject: '2021-01-28T00:00:00.000Z'
}
{
 pageNumber: 0,
 reconstructed: false,
 value: '2021-01-28',
 probability: 0.94,
 bbox: [
 [ 0.394, 0.448 ],
 [ 0.474, 0.448 ],
 [ 0.474, 0.464 ],
 [ 0.394, 0.464 ]
 ],
 dateObject: '2021-01-28T00:00:00.000Z'
}
</code></pre>

&nbsp;

&nbsp;

<h2>Supplier information</h2>

&nbsp;

invoice.supplier:&nbsp;Supplier name as written in the invoice

invoice.paymentDetails:&nbsp;List of Supplier&#39;s payment details written in the invoice

invoice.companyNumber:&nbsp;Supplier&#39;s company number

&nbsp;

<pre>
<code class="language-javascript">mindeeClient.invoice.parse(
 "invoice.png",
 'path',
)
 .then((res) =&gt; {
 console.log(res.invoice.supplier);
 console.log(res.invoice.paymentDetails);
 console.log(res.invoice.companyNumber);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

<pre>
<code class="language-json">{
 pageNumber: 0,
 reconstructed: false,
 value: 'Google',
 probability: 0.18,
 bbox: [
 [ 0.125, 0.074 ],
 [ 0.172, 0.074 ],
 [ 0.172, 0.083 ],
 [ 0.125, 0.083 ]
 ]
}
[
 {
 pageNumber: 0,
 reconstructed: false,
 value: 'EE657700xxxxxx265345',
 probability: 0.97,
 bbox: [ [Array], [Array], [Array], [Array] ],
 iban: 'EE6xxxxxxxxxx345',
 swift: 'LxxxxxE22'
 }
]
[
 {
 pageNumber: 0,
 reconstructed: false,
 value: '513xxxx80',
 probability: 0.9,
 bbox: [ [Array], [Array], [Array], [Array] ],
 type: 'NIF'
 },
 {
 pageNumber: 0,
 reconstructed: false,
 value: 'Pxxxxxx004',
 probability: 0.99,
 bbox: [ [Array], [Array], [Array], [Array] ],
 type: 'VAT NUMBER'
 }
]
</code></pre>

&nbsp;

&nbsp;

<h2>Locale and currency</h2>

&nbsp;

locale: Language ISO code

Contains an extra attribute:

&nbsp;

<ul>
	<li>currency: (String), ISO currency code</li>
</ul>

&nbsp;

<pre>
<code class="language-javascript">mindeeClient.invoice.parse(
 "invoice.png",
 'path',
)
 .then((res) =&gt; {
 console.log(res.invoice.locale);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

<pre>
<code class="language-json">{
 pageNumber: 0,
 reconstructed: false,
 probability: 0,
 bbox: [],
 language: 'pt',
 currency: 'EUR'
}
</code></pre>

&nbsp;

&nbsp;

<h2>InvoiceNumber</h2>

&nbsp;

<pre>
<code class="language-javascript">mindeeClient.invoice.parse(
 "invoice.png",
 'path',
)
 .then((res) =&gt; {
 console.log(res.invoice.invoiceNumber);
 })
 .catch((err) =&gt; {
 console.error(err);
 });</code></pre>

&nbsp;

<pre>
<code class="language-json">{
 pageNumber: 0,
 reconstructed: false,
 value: 'Fxxxxxx1/4',
 probability: 0.98,
 bbox: [
 [ 0.805, 0.09 ],
 [ 0.954, 0.09 ],
 [ 0.954, 0.112 ],
 [ 0.805, 0.112 ]
 ]
}
</code></pre>

&nbsp;

&nbsp;

Access invoice fields using Mindee OCR API and node.js

Invoice fields

Parsing Invoices

<h1>Parsing international passports with Mindee&#39;s API</h1>

&nbsp;

Many onboarding processes in mobile or web apps require to extract some data from ID documents. In this tutorial, you will learn how to automatically extract data from passports to offer to your users the best onboarding experience.

&nbsp;

We will walk through the steps to use Mindee&rsquo;s Internation Passport&nbsp;Parsing API.&nbsp; Let&rsquo;s get started!&nbsp;

&nbsp;

<h2>API Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free Mindee account. <a href="https://platform.mindee.net/signup" target="_blank">Sign up</a>&nbsp;and confirm your email to login.</li>
	<li>A picture&nbsp;of a passport.&nbsp; You can use yours&nbsp;safely as data protection is one of our priority and we won&#39;t send your images to any third party application. You can also download a fake one <a href="https://mindee-public-website.s3.amazonaws.com/blog/2020/09/03/fake_passport.jpg">here</a>.</li>
</ol>

&nbsp;

&nbsp;

<img alt="" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2020/08/07/passport_pic.jpg" />

&nbsp;

&nbsp;

<h2>Setting up the API</h2>

&nbsp;

Log into your Mindee account and access your passport API environment by clicking the International&nbsp;Passport&nbsp;card:

&nbsp;

&nbsp;

<img alt="passport api mindee" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2020/08/07/passport_article.jpg" />

&nbsp;

To activate the API, click the &ldquo;Try for Free&rdquo; button. This will give you access the API for free 50 times a month.&nbsp; There are four sections on the API landing page, noted in the left navigation. You are currently on the dashboard, and there&nbsp;are additional links to &ldquo;Documentation&rdquo;, &ldquo;Credentials&rdquo; and &ldquo;Live Interface&rdquo;.&nbsp; The docs tab has all of the technical details you&rsquo;ll need to build for the passports API endpoint, and the Live Interface is a cool interactive demo.

We&#39;ll start by clicking &quot;credentials&quot; and creating a new API token, in this case, named Tutorial:

&nbsp;

<img src="https://mindee-public-website.s3.amazonaws.com/blog/2020/09/01/pasted-image-0-1.png" style="height:198px; width:400px" />

&nbsp;

Click &ldquo;Add New Key&rdquo; and you&rsquo;ll be able to see your API token,.

&nbsp;

If you move back to the documentation tab, you can pick your API token, and the language, and create an API call.&nbsp; Here is the API call in cURL:

<pre>
<code class="language-bash">curl -X POST \ https://api.mindee.net/products/passport/v1/predict \ 
-H 'X-Inferuser-Token: {apiToken}’ \
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
-F file=@/path/to/your/file.png</code></pre>

&nbsp;

Simply replace {apiToken}&nbsp;with your new API token and /path/to/your/file/png with the path to your receipt.&nbsp;

 
That&#39;s it! We&#39;re all set to run this command in the terminal, and see the data that Mindee can extract.

In this tutirla, I&#39;ll use a sample [Cyprus passport](https://commons.wikimedia.org/wiki/File:Cyprus_passport_data_page.jpg) from Wikimedia.

<img alt="Cyprus Passport used in this example" src="https://mindee-public-website.s3.amazonaws.com/blog/2020/09/04/1024px-cyprus_passport_data_page.jpg" style="height:334px; width:514px" />

<h2>&nbsp;</h2>

<h2>Extracted fields</h2>

&nbsp;

The JSON response can be a bit verbose, so we will break it into sections to describe what we are seeing.&nbsp; The initial sections describe the API and the API response:

<pre>
<code class="language-json">"call": {
		"endpoint": {
			"name": "passport",
			"version": "1.0"
		},
		"finished_at": "2020-09-04T20:57:51+00:00",
		"id": "7d81f5fe-fce1-4b63-b3de-cc5948c39b55",
		"n_documents": 1,
		"n_inputs": 1,
		"processing_time": 1.988,
		"started_at": "2020-09-04T20:57:49+00:00"
	},
	"documents": [
		{
			"id": "57aded44-a216-45be-859c-f949cc5b011d",
			"name": "1024px-Cyprus_passport_data_page.jpg"
		}
	],</code></pre>

Here, we can see that passport API was called, and it processed one document in about 2 seconds.&nbsp; What we are really interested in seeing is the results, and these can be found in the predictions section of the JSON file:

<h2>&nbsp;</h2>

<h2>Predictions</h2>

&nbsp;

The API response is in alphabetical order of items extracted from the document:

&nbsp;

<h3>Birth Date</h3>

<pre>
<code class="language-json">"birth_date": {
				"probability": 1.0,
				"segmentation": {
					"bounding_box": [
						[.053,	0.912],
						[0.955,0.912],
						[0.955,0.969],
						[0.053,0.969]
					]
				},
				"value": "1970-01-01"
			},</code></pre>

The API matched the brithdate in the MRX fields (more on that in a minute), so is 100% confident that the birthdate is 1 January, 1970. The bounding box gives four (x,y) points in the image where the date of birth can be found.

&nbsp;

<h3>Country of issuance</h3>

The API then attempts to parse the country that issued the passport.&nbsp;

<pre>
<code class="language-json">"country": {
				"probability": 0.99,
				"segmentation": {
					"bounding_box": [
						[0.479,0.199],
						[0.522,0.199],
						[0.522,0.234],
						[0.479,0.234]
					]
				},
				"value": "CYP"
			},</code></pre>

&nbsp;

The API is 99% certain that Cyprus is the issuing country. This is really incredible, as the API team tells me that there are no Cyprus passports in the data training set. Again, 4 (x,y) coordinates in the image point out where CYP is used.

<h3>&nbsp;</h3>

<h3>Expiration Date</h3>

<pre>
<code class="language-json">"expiry_date": {
				"probability": 1.0,
				"segmentation": {
					"bounding_box": [
						[0.053,0.912],
						[0.955,0.912],
						[0.955,0.969],
						[0.053,0.969]
					]
				},
				"value": "2020-12-01"
			},</code></pre>

Many countries do not allow entry with under six months of validity left on the passport.&nbsp; This woman should probably start the renewal process,&nbsp; as the API has identified that this passport will expire on 1 December, 2020.

<h3>&nbsp;</h3>

<h3>Gender</h3>

<pre>
<code class="language-json">			"gender": {
				"probability": 0.1,
				"segmentation": {
					"bounding_box": [
						[0.053,0.912],
						[0.955,0.912],
						[0.955,0.969],
						[0.053,0.969]
					]
				},
				"value": "F"
			},</code></pre>

The API&#39;s confidence is not high on this one, just 10%, but it does correctly identify the gender as female.

<h3>&nbsp;</h3>

<h3>Given Name</h3>

<pre>
<code class="language-json">			"given_names": [
				{
					"probability": 0.99,
					"segmentation": {
						"bounding_box": [
							[0.048,0.844],
							[0.952,0.844],
							[0.952,0.895],
							[0.048,0.895]
						]
					},
					"value": "AFRODITI"
				}
			],</code></pre>

The API is really certain (99% confidence) that the given name on the passport is Afroditi, meaning that our passport holder was named after the Greek goddess of beauty and love.

<h3>&nbsp;</h3>

<h3>ID Number</h3>

<pre>
<code class="language-json">			"id_number": {
				"probability": 1.0,
				"segmentation": {
					"bounding_box": [
						[0.673,0.197],
						[0.774,0.197],
						[0.774,0.233],
						[0.673,0.233]
					]
				},
				"value": "K00000413"
			},</code></pre>

While we are not numbers, in many ways, we are identified by numbers. In this case, the passport ID number, which must be used when booking flights, checking into hotels, etc.&nbsp; The API has found the location (marked by the 4 (x,y) points in the bounding box, for the Passport number, which is K00000413.

&nbsp;

<h3>Issuance Date</h3>

Contining the alphabetical trip through the passport fields, we next come to the date the passport was issued:

&nbsp;

<pre>
<code class="language-json">			"issuance_date": {
				"probability": 0.18,
				"segmentation": {
					"bounding_box": [
						[0.353,0.586],
						[0.426,0.586],
						[0.426,0.62],
						[0.353,0.62]
					]
				},
				"value": "2014-01-21"
			},</code></pre>

In this case, the API missed the issuance date.&nbsp; However, while our API does miss in it&#39;s predictions, in this case, the issue is due to the fact that the passport is a sample (we&#39;ll have a blog post on this soon).&nbsp; The issuance date is missing from the MRZ zone of the sample&nbsp;passport, so the API is unable to accurately measure the issuance date.

&nbsp;

<h3>Surname</h3>

<pre>
<code class="language-json">"surname": {
				"probability": 0.98,
				"segmentation": {
					"bounding_box": [
						[0.33,0.272],
						[0.444,0.272],
						[0.444,0.309],
						[0.33,0.309]
					]
				},
				"value": "ANONYMOU"
			}</code></pre>

Our fake passport holder has a surname of Anonymou, which is correctly identified by the API, along with bounding boxes.

&nbsp;

<h3>MRZ1 and MRZ2</h3>

The last two regions identified by the API (which I pulled *slightly* out of alphabetical order for readability) are the 2 Machine Readable Zone (MRZ) lines:

<pre>
<code class="language-json">"mrz1": {
				"probability": 0.99,
				"segmentation": {
					"bounding_box": [
						[0.048,0.844],
						[0.952,0.844],
						[0.952,0.895],
						[0.048,0.895]
					]
				},
				"value": "P&lt;CYPANONYMOU&lt;&lt;AFRODITI&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;"
			},
"mrz2": {
				"probability": 0.1,
				"segmentation": {
					"bounding_box": [
						[0.053,0.912],
						[0.955,0.912],
						[0.955,0.969],
						[0.053,0.969]
					]
				},
				"value": "K000004134CYP7001017F2012010&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;4"
			},</code></pre>

The MRZ zones (if you cannot tell from the JSON above) are the two lines of machine readable text at the bottom of every passport page.&nbsp; MRZ1 is line one, and MRZ2 is line 2.&nbsp; Most of the identifying information can be pulled from this area.&nbsp; The confidence interval for line one is high, as the fake passport has this area filled out correctly.&nbsp; However, the confidence level is low, as the algorithm expected more characters other than &quot;&lt;&quot; to supply the issuance date.

&nbsp;

<h2>Conclusion</h2>

&nbsp;

And there you have it! Mindee&#39;s passport API parsed a passport in under 2 seconds, matching nearly all of the sections of the passport quickly and accurately. Give it&nbsp;try with our free tier, and let us know in the chat how it works for you!

Many onboarding processes in mobile or web apps require to extract some data from ID documents. In this tutorial, you will learn how to automatically extract data from passports to offer to your users the best onboarding experience.

<h1>Extract data from passports with Mindee&#39;s API using Python&nbsp;</h1>

&nbsp;

In this tutorial, you will learn how to&nbsp;extract data from passports (pdfs or images)&nbsp;using python. This can help you automatically process id document for user onboarding or KYC processes.

&nbsp;

If you want to get more details about how to use the mindee python library for passports, please see:

<ul>
	<li>1.&nbsp;<a href="https://mindee.com/documentation/apis/tutorial-parsing-passports-with-mindees-api/passport-python/getting-started">Getting started with passport parsing in python</a></li>
	<li>2.&nbsp;<a href="https://mindee.com/documentation/apis/tutorial-parsing-passports-with-mindees-api/passport-python/passport-fields">Extracted passport fields</a></li>
	<li>3.&nbsp;<a href="https://mindee.com/documentation/apis/tutorial-parsing-passports-with-mindees-api/passport-python/save-and-restore">Save and restore passport API responses</a></li>
	<li>4.&nbsp;<a href="https://mindee.com/documentation/apis/tutorial-parsing-passports-with-mindees-api/passport-python/passport-data-validation">Passport data validations</a></li>
	<li>5.&nbsp;<a href="https://mindee.com/documentation/apis/tutorial-parsing-passports-with-mindees-api/passport-python/performances-benchmark">Benchmark Passport OCR performances</a></li>
</ul>

&nbsp;

&nbsp;

<h2>Tutorial</h2>

&nbsp;

In this tutorial, you will learn how to extract passport data automatically using Mindee python helper library,&nbsp;transform a pdf into an image file, and how to&nbsp;highlight features&nbsp;on it to make human validation easier.&nbsp;

&nbsp;

The final result looks like this:

&nbsp;

<img alt="" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2020/11/04/ezgifcom-video-to-gif-17.gif" />

&nbsp;

&nbsp;

<h2>API Prerequisites</h2>

&nbsp;

<ol>
	<li>You&rsquo;ll need a free Mindee account.&nbsp;<a href="https://platform.mindee.net/signup">Sign up</a>&nbsp;and confirm your email to login.</li>
	<li>A passport image or pdf. You can&nbsp;download one&nbsp;<a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2020/11/04/fake_passport-1.jpg" target="_blank">here</a>&nbsp;or&nbsp;<a href="https://www.google.com/search?q=fake+passport">check out on google image</a></li>
</ol>

&nbsp;

&nbsp;

<h2>Setup the project</h2>

&nbsp;

Create an empty directory wherever you want in your laptop, we&rsquo;ll call ours &ldquo;python_passport&rsquo;, and create a &lsquo;main.py&rsquo; file in it.

&nbsp; 
We recommend you to create a virtual environment before going to the next step, in order to avoid installing the required packages globally on your machine.To do so, run the following command:

&nbsp;

(macOS and Linux)

<pre>
<code class="language-bash">python -m venv venv</code></pre>

 
(windows)

<pre>
<code class="language-bash">py -m venv venv</code></pre>

&nbsp;

Then, you&#39;ll need to activate your venv, so that pip will install packages in that folder.

&nbsp;

(macOS and Linux)

<pre>
<code class="language-bash">source venv/bin/activate</code></pre>

 
(Windows)

<pre>
<code class="language-bash">.\env\Scripts\activate</code></pre>

&nbsp;

&nbsp;

We need now to Install mindee python library&nbsp;from PyPi using pip, a package manager for Python.

&nbsp;

<pre>
<code class="language-bash">pip install mindee</code></pre>

&nbsp;

If the install throws an error, you might need to upgrade your version of pip. To do so, run:

&nbsp;

<pre>
<code class="language-bash">pip install --upgrade pip </code></pre>

&nbsp;

Don&#39;t have pip installed? Try installing it, by running this from the command line:

&nbsp;

<pre>
<code class="language-bash">$ curl https://bootstrap.pypa.io/get-pip.py | python</code></pre>

&nbsp;

That&rsquo;s it for the setup, let&rsquo;s call the API.

&nbsp;

&nbsp;

<h2>Call the Mindee Passport API</h2>

&nbsp;

&nbsp;

Login onto the <a href="https://platform.mindee.net/">Mindee dashboard</a>&nbsp;and enter the&nbsp;Passport API&nbsp;environment by clicking the following card.

&nbsp;

&nbsp;

<img alt="mindee invoice api card" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2020/11/04/passport_article.jpg" />

&nbsp;

&nbsp;

If you don&rsquo;t have an API token yet for this API, go to the &ldquo;credentials&rdquo; section and create a new API Key.&nbsp;

&nbsp;

&nbsp;

<img alt="new api token" src="https://lh3.googleusercontent.com/CWRZZFfY2uVd5Lyx-wxxR_EjWcw6ZBryFhhpNMwP1DBO5LQEEUBfqos2sPjCE4BQ6AdxelSzaybQEl5EGGXDL-eqwr5qXA3dtgn8_TquDVfVp8xFMsjV_npDbTblZ-wXTgOkwoEy" />

&nbsp;

&nbsp;

Open the &ldquo;main.py&rdquo; file you created a few minutes ago, and paste the following&nbsp;code:

&nbsp;

&nbsp;

<pre>
<code>from mindee import Client

filepath = "./path/to/passport.jpg"
passport_token = "my-token-here"

mindee_client = Client(passport_token=passport_token)

if __name__ == "__main__":

 response = mindee_client.parse_passport(filepath)

 print(response.passport)</code></pre>

&nbsp;

&nbsp;

Replace the &ldquo;path/to/my/file&rdquo; placeholder in the code with the path of the passport pdf or image you want to parse. Replace the &ldquo;my-token-here&rdquo; with the API token you created previously in the platform.

&nbsp;

&nbsp;

You can get back to your console and run the script

&nbsp;

&nbsp;

<pre>
<code>python main.py</code></pre>

&nbsp;

&nbsp;

You should see the details of the passport printed in your console:

&nbsp;

<pre>
<code class="language-apache">-----Passport data-----
Filename: passport.jpg 
Full name: HENERT PUDARSAN 
Given names: HENERT 
Surname: PUDARSAN
Country: GBR
ID Number: 707797979
Issuance date: 2012-04-22
Birth date: 1995-05-20
Expiry date: 2017-04-22
MRZ 1: P&lt;GBRPUDARSAN&lt;&lt;HENERT&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;
MRZ 2: 7077979792GBR9505209M1704224&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;00
MRZ: P&lt;GBRPUDARSAN&lt;&lt;HENERT&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;7077979792GBR9505209M1704224&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;00
----------------------</code></pre>

&nbsp;

Now let&rsquo;s take a look at the passport object.

&nbsp;

<h2>&nbsp;</h2>

<h2>Extracted passport data</h2>

&nbsp;

The API returns a list of different fields in the passport (given names, surname, date of birth &hellip;). You can find this list in the <a href="https://mindee.com/documentation/apis/tutorial-parsing-passports-with-mindees-api/passport-python">Extracted passport fields</a> documentation

&nbsp;

The passport object under the&nbsp;<code>Response.passport</code>&nbsp;attribute has all the features you need to get the passport data and coordinates in the image to highlight the results.

&nbsp;

Here is a piece of code to show you how to get some of them:

&nbsp;

<pre>
<code class="language-python">from mindee import Client

filepath = "./path/to/passport.jpg"
passport_token = "my-token-here"

mindee_client = Client(passport_token=passport_token)

if __name__ == "__main__":
 response = mindee_client.parse_passport(filepath)

 print(response.passport)

 # To get the list of names
 print(response.passport.given_names)

 # Loop on each given name
 for given_name in response.passport.given_names:
 # To get the name string
 print(given_name.value)

 # To get the passport's owner surname (string)
 print(response.passport.surname.value)

 # To get the passport's owner gender (string among {"M", "F"}
 print(response.passport.gender.value)

 # To get the passport's owner full name (string)
 print(response.passport.full_name.value)

 # To get the passport's owner birth place (string)
 print(response.passport.birth_place.value)

 # To get the passport's owner date of birth (string)
 print(response.passport.birth_date.value)

 # To get the passport expiry date (string)
 print(response.passport.expiry_date.value)

 # To get the passport date of issuance (string)
 print(response.passport.issuance_date.value)

 # To get the passport first line of machine readable zone (string)
 print(response.passport.mrz1.value)

 # To get the passport second line of machine readable zone (string)
 print(response.passport.mrz2.value)

 # To get the passport full machine readable zone (string)
 print(response.passport.mrz.value)

 # To get the passport id number (string)
 print(response.passport.id_number.value)

 # To get the passport country code (string)
 print(response.passport.country_code.value)</code></pre>

&nbsp;

Run the script and check out the results in your console!

&nbsp;&nbsp;

Now we are going to add a piece of code that will highlight the features extracted on the pdf. It can help you or your users to very quickly validate the data extraction.

&nbsp;

&nbsp;

<h2>Convert pdf into an image</h2>

&nbsp;

If you&rsquo;re using a pdf,&nbsp;we need first to convert the pdf into an image, as it&rsquo;s more convenient to use for highlighting the features on it. If your passport is an image, you can skip this part.

&nbsp;

To convert the pdf into a jpeg we are going to use the&nbsp;<a href="https://github.com/Belval/pdf2image">pdf2image</a>&nbsp;python library; First install it in your environment:

&nbsp;

<pre>
<code>pip install pdf2image</code></pre>

&nbsp;

Important note: Dealing with pdfs is not easy and this library might not work directly because it sometimes requires some extra dependencies. Check out the github to make sure you installed it correctly.

&nbsp;

Once installed, we are going to create an image out of our pdf:

&nbsp;

&nbsp;

<pre>
<code>from pdf2image import convert_from_path, convert_from_bytes
images = convert_from_path(filepath)</code></pre>

&nbsp;

&nbsp;

The images object is an array of Pillow images. Pillow is an image library for python. As we want to use openCV for highlighting the features on the image, we need to convert the pillow image into an OpenCV image. OpenCV is based on the numpy library, and to convert it we are going to use numpy directly.

&nbsp;

If you don&rsquo;t have numpy installed on your environment:

&nbsp;

&nbsp;

<pre>
<code>pip install numpy</code></pre>

&nbsp;

&nbsp;

Now let&rsquo;s convert the Pillow image into an opencv format using numpy:

&nbsp;

&nbsp;

<pre>
<code>import numpy as np
from pdf2image import convert_from_path
 
def pdf_to_cv_image(filepath, page=0):
 image = convert_from_path(filepath)[page]
 return np.asarray(image)</code></pre>

&nbsp;

 
 
&nbsp;

<h2>Highlight features on the image</h2>

&nbsp;

Let&rsquo;s try to highlight the features as if someone did it with a pen.

&nbsp;

First, you&rsquo;ll need to install the computer vision python library OpenCV if you don&rsquo;t have it already installed in your env. To do so, run:

&nbsp;

&nbsp;

<pre>
<code>pip install opencv-python</code></pre>

&nbsp;

Each field extracted in the&nbsp;<code>Response.passport</code>&nbsp;object has a&nbsp;<code>Field.bbox</code>&nbsp;object. It&#39;s as simple list of bounding boxes relative vertices (% of the image width and height).

&nbsp;

Check out what&#39;s inside for the total_incl.bbox for example, doing:

&nbsp;

<pre>
<code>print(passport_data.passport.mrz1.bbox)</code></pre>

&nbsp;

Now that we know how to access the bounding boxes coordinates, we are going to create our&nbsp;highlighter function in which we can feed any field bounding boxes&nbsp;in different steps, as follows:

&nbsp;

<ol>
	<li>Check if the file is a pdf or an image and load it as an openCV image in both scenarios</li>
	<li>Create a mask image</li>
	<li>Loop on each feature coordinates and draw the feature rectangle on our mask</li>
	<li>Overlay the mask and original image with alpha</li>
	<li>Display image to the user</li>
</ol>

&nbsp;

Note: each coordinate returned by the API is relative (in % of the image). You&#39;ll see there is a relative to absolute conversion in the code.

&nbsp;

Here is the code step by step:

&nbsp;

<pre>
<code>def highlight_features(img_path, coordinates):
 # step 1: Check if the file is a pdf or an image and load it as an openCV image in both scenarios
 if img_path.endswith(".pdf"):
 cv_image = pdf_to_cv_image(img_path)
 else:
 cv_image = cv2.imread(img_path)

 # step 2: create mask image
 overlay = cv_image.copy()
 h, w = cv_image.shape[:2]

 # step 3: Loop on each feature coordinates and draw the feature rectangle on our mask
 for coord in coordinates:
 if len(coord):
 # Create points absolute coordinates
 pt1 = (int(w*coord[0][0]), int(h*coord[0][1]))
 pt2 = (int(w*coord[2][0]), int(h*coord[2][1]))
 cv2.rectangle(overlay, pt1, pt2, (70, 230, 244), cv2.FILLED)

 # step 4: Overlay the mask and original image with alpha
 final_image = cv2.addWeighted(overlay, 0.5, cv_image, 0.5, 0)

 # step 5: Display image to the user
 cv2.imshow("highlghted_image", cv2.resize(final_image, (600, int(600*h/w))))
 cv2.waitKey(0)</code></pre>

 
&nbsp;

Finally, we just have to modify a bit the code for&nbsp;executing the highlighting function and printing the results. Here is what the final code:

&nbsp;

&nbsp;

<pre>
<code>import cv2
import numpy as np
from pdf2image import convert_from_path
from mindee import Client


filepath = "./path/to/passport.jpg"
passport_token = "my-token-here"


def pdf_to_cv_image(filepath, page=0):
 image = convert_from_path(filepath)[page]
 return np.asarray(image)


def highlight_features(img_path, coordinates):
 # step 1: Check if the file is a pdf or an image and load it as an openCV image in both scenarios
 if img_path.endswith(".pdf"):
 cv_image = pdf_to_cv_image(img_path)
 else:
 cv_image = cv2.imread(img_path)

 # step 2: create mask image
 overlay = cv_image.copy()
 h, w = cv_image.shape[:2]

 # step 3: Loop on each feature coordinates and draw the feature rectangle on our mask
 for coord in coordinates:
 if len(coord):
 pt1 = (int(w * coord[0][0]), int(h * coord[0][1]))
 pt2 = (int(w * coord[2][0]), int(h * coord[2][1]))
 cv2.rectangle(overlay, pt1, pt2, (70, 230, 244), cv2.FILLED)

 # step 4: Overlay the mask and original image with alpha
 final_image = cv2.addWeighted(overlay, 0.5, cv_image, 0.5, 0)

 # step 5: Display image to the user
 cv2.imshow("highlghted_image", cv2.resize(final_image, (600, int(600 * h / w))))
 cv2.waitKey(0)


mindee_client = Client(passport_token=passport_token)

if __name__ == "__main__":
 response = mindee_client.parse_passport(filepath)

 print(response.passport)

 highlight_features(
 filepath,
 [
 response.passport.surname.bbox,
 response.passport.given_names[0].bbox,
 response.passport.mrz1.bbox,
 response.passport.mrz2.bbox,
 response.passport.birth_place.bbox,
 response.passport.country.bbox,
 response.passport.issuance_date.bbox,
 response.passport.expiry_date.bbox,
 response.passport.birth_date.bbox,
 response.passport.id_number.bbox
 ]
 )</code></pre>

 
&nbsp;

And the final result!

&nbsp;

<img alt="" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2020/11/04/highlghted_image.jpg" /> 
&nbsp;

&nbsp;

&nbsp;

<h2>Conclusion</h2>

&nbsp;

And there you have it! Mindee&#39;s passport API parsed a passport in under 2 seconds, matching nearly all of the sections of the passport quickly and accurately. Give it&nbsp;try with our free tier, and let us know in the chat how it works for you!

<h1>Getting started</h1>

<h2>&nbsp;</h2>

&nbsp;

Get started with passport parsing in python.

&nbsp;

<h2>1. Install the mindee python helper library</h2>

&nbsp;

Install from PyPi using&nbsp;<a href="https://pip.pypa.io/en/latest/" rel="nofollow">pip</a>, a package manager for Python.

&nbsp;

<pre>
<code class="language-bash">pip install mindee</code></pre>

&nbsp;

Don&#39;t have pip installed? Try installing it, by running this from the command line:

&nbsp;

<pre>
<code class="language-bash">$ curl https://bootstrap.pypa.io/get-pip.py | python</code></pre>

&nbsp;

Getting started with the Mindee API couldn&#39;t be easier. Create a&nbsp;<code>Client</code>&nbsp;and you&#39;re ready to go.

&nbsp;

&nbsp;

&nbsp;

<h2>2. Instantiate your Client</h2>

&nbsp;

The&nbsp;<code>mindee.Client</code>&nbsp;needs your API credentials. You can either pass these directly to the constructor (see the code below) or via environment variables.

&nbsp;

Depending on what type of document you want to parse, you need to add specifics auth token for each endpoint.

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(
 passport_token="your_passport_api_token_here",
 raise_on_error=True
)</code></pre>

&nbsp;

<ul>
	<li>passport_token:&nbsp;(string) API key for passports endpoint</li>
	<li>raise_on_error: (boolean, default True) Specify wheter or not raising an Exception when HTTP errors occur</li>
</ul>

&nbsp;

&nbsp;

We suggest storing your credentials as environment variables. Why? You&#39;ll never have to worry about committing your credentials and accidentally posting them somewhere public.

&nbsp;

&nbsp;

&nbsp;

<h2>3. Parse a Passport</h2>

&nbsp;

Using the parse_passport method from your client object, you can pass any image or pdf file (see <a href="https://mindee.com/documentation/apis-overview/input-data">inputs file</a>&nbsp;for pdf pages number limits) to get the passport data.

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(
 passport_token="your_passport_api_token_here",
 raise_on_error=True
)

parsed_data = mindee_client.parse_passport("/path/to/file")
</code></pre>

&nbsp;

&nbsp;

<h3>Input types</h3>

&nbsp;

You can pass your input file in three ways:

&nbsp;

From file path

<pre>
<code class="language-python">passport_data = mindee_client.parse_passport('/path/to/file', input_type="path")</code></pre>

&nbsp;

&nbsp;From a file object

<pre>
<code class="language-python">with open('/path/to/file', 'rb') as fp:
 passport_data = mindee_client.parse_passport(fp, input_type="file")
</code></pre>

&nbsp;

From a base64

<pre>
<code class="language-python">passport_data = mindee_client.parse_passport(base64_string, input_type="base64")</code></pre>

&nbsp;

&nbsp;

passport_data structure

&nbsp;

The passport_data object returned by the&nbsp;<code>parse_passport</code>&nbsp;method contains the following elements:

&nbsp;

passport_data.passport

&nbsp;

The passport attribute&nbsp;is the Passport object constructed by gathering all the pages into a single document.

&nbsp;

<pre>
<code class="language-python">passport_data.passport # returns a unique object from class Passport</code></pre>

<h4>&nbsp;</h4>

&nbsp;

passport_data.passports

&nbsp;

For multi pages pdf, the &#39;pages&#39; attribute is a list of Passport objects, each object is constructed using a unique page of the pdf

&nbsp;

<pre>
<code class="language-python">passport_data.passports # [Passport, Passport ...] </code></pre>

<h4>&nbsp;</h4>

&nbsp;

passport_data.http_response

&nbsp;

Contains the full Mindee API HTTP response object

&nbsp;

<pre>
<code class="language-python">passport_data.http_response # full HTTP request object </code></pre>

&nbsp;

&nbsp;

&nbsp;

<h2>4. Display the results</h2>

&nbsp;

You only have to print your passport object to display the different extracted fields:

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(
 passport_token="your_passport_token_here",
 raise_on_error=True
)

passport_data = mindee_client.parse_passport("/path/to/passport")

print(passport_data.passport)</code></pre>

&nbsp;

&nbsp;

Get started with Passport data extraction using Mindee python library helper.

<h1>Passport fields</h1>

&nbsp;

&nbsp;

<h2>General fields structure</h2>

&nbsp;

Each Passport&nbsp;object contains a set of different fields.

&nbsp;

To acces an passport object, you need to create a <code>mindee.Client&nbsp;</code>and call the <code>Client.parse_passport</code>&nbsp;method:

&nbsp;

<pre>
<code>from mindee import Client

# Instantiate your client
mindee_client = Client(passport_token="your_token_here")

# Call the Mindee passport endpoint and parse the result
passport_data = mindee_client.parse_passport("/path/to/my/file")</code></pre>

&nbsp;

Whatever the field type, each of them contains the four following attributes:

&nbsp;

<ul>
	<li>Value: (Str or Float depending on the field type), corresponds to the field value. Set to None if the field was not extracted.</li>
	<li>Probability: (Float), confidence score of the field prediction.</li>
	<li>bbox:(Array[Float]), contains the relative vertices coordinates of the bounding box containing the field in the image. If the field is not written, the bbox is an empty array.&nbsp;</li>
	<li>reconstructed: (Bool), True if the field was reconstructed using other fields.</li>
</ul>

&nbsp;

Depending on the Field type, there can be extra attributes.

&nbsp;

&nbsp;

&nbsp;

<h2>Passport&#39;s owner data</h2>

&nbsp;

passport.given_names:&nbsp;List of passport&#39;s owner given names

<pre>
<code class="language-python"># To get the list of names
given_names = passport_data.passport.given_names

# Loop on each given name
for given_name in given_names:

 # To get the name string
 name = given_name.value</code></pre>

&nbsp;

passport.surname:&nbsp;Passport&#39;s owner surname

<pre>
<code class="language-python"># To get the passport's owner surname (string)
surname = passport_data.passport.surname.value</code></pre>

&nbsp;

passport.gender:&nbsp;Passport&#39;s owner gender (M / F)

<pre>
<code class="language-python"># To get the passport's owner gender (string among {"M", "F"}
gender = passport_data.passport.gender.value</code></pre>

&nbsp;

passport.full_name:&nbsp;Reconstructed&nbsp;Passport&#39;s owner full name from surname and given_names

<pre>
<code class="language-python"># To get the passport's owner full name (string)
full_name = passport_data.passport.full_name.value</code></pre>

&nbsp;

passport.birth_place:&nbsp;Passport&#39;s owner birth place

<pre>
<code class="language-python"># To get the passport's owner birth place (string)
birth_place = passport_data.passport.birth_place.value</code></pre>

<h2>&nbsp;</h2>

<h2>&nbsp;</h2>

<h2>Dates</h2>

&nbsp;

Each date field comes with an extra attribute:

&nbsp;

<ul>
	<li>date_object: (Datetime), datetime object from python datetime.date library</li>
</ul>

&nbsp;

&nbsp;

passport.birth_date:&nbsp;Passport&#39;s owner date of birth

<pre>
<code class="language-python"># To get the passport's owner date of birth (string)
birth_date = passport_data.passport.birth_date.value</code></pre>

&nbsp;

passport.expiry_date:&nbsp;Passport expiry date<img src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2020/11/03/image-20201103134559-1.gif" />

<pre>
<code class="language-python"># To get the passport expiry date (string)
expiry_date = passport_data.passport.expiry_date.value</code></pre>

&nbsp;

passport.issuance_date:&nbsp;Passport date of issuance

<pre>
<code class="language-python"># To get the passport date of issuance (string)
issuance_date = passport_data.passport.issuance_date.value</code></pre>

&nbsp;

&nbsp;

<h2>Passports metadata</h2>

&nbsp;

passport.mrz1:&nbsp;Passport first line of machine readable zone

<pre>
<code class="language-python"># To get the passport first line of machine readable zone (string)
mrz1 = passport_data.passport.mrz1.value</code></pre>

&nbsp;

passport.mrz2:&nbsp;Passport second line of machine readable zone

<pre>
<code class="language-python"># To get the passport second line of machine readable zone (string)
mrz2 = passport_data.passport.mrz2.value</code></pre>

&nbsp;

passport.mrz: Reconstructed passport full machine readable zone from mrz1 and mrz2

<pre>
<code class="language-python"># To get the passport full machine readable zone (string)
mrz = passport_data.passport.mrz.value</code></pre>

&nbsp;

passport.id_number:&nbsp;Passport identification number

<pre>
<code class="language-python"># To get the passport id number (string)
id_number = passport_data.passport.id_number.value</code></pre>

&nbsp;

passport.country:&nbsp;Passport country code

<pre>
<code class="language-python"># To get the passport country code (string)
country_code = passport_data.passport.country_code.value</code></pre>

&nbsp;

How to use extracted passport field using Mindee python helper library.

Passport fields

<h1>Save and restore</h1>

&nbsp;

The&nbsp;<code>mindee.Client</code>&nbsp;includes methods for saving and restoring api calls. Those methods are helpful if you want to perform tests or if you simply want to reload passport&nbsp;objects corresponding to your files without performing any additional request.

&nbsp;

&nbsp;

<h2>Save a call</h2>

&nbsp;

The <code>mindee.Response</code> object returned by the <code>mindee.Client</code> when parsing a file is responsible for wrapping all the data from Mindee API calls. It&nbsp;includes a <code>dump</code> method that can be used as follows:

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(passport_token="your_passport_token_here")

passport_data = mindee_client.parse_passport("./passport.jpg")

passport_data.dump("./passport_dump.json")</code></pre>

&nbsp;

&nbsp;

<h2>Restore a call</h2>

&nbsp;

To restore a client response object, you can use the <code>mindee.Response</code> as follows:

&nbsp;

<pre>
<code class="language-python">from mindee import Response

passport_data = Response.load("./passport_dump.json")</code></pre>

&nbsp;

&nbsp;

Save and restore Mindee passport API response using python helper library.

<h1>Passport data validation</h1>

&nbsp;

&nbsp;

Each Passport object comes with a set of validations checks. it can be accessed under the attribute <code>Passport.checklist</code>.

&nbsp;

Example:

&nbsp;

<pre>
<code class="language-python">from mindee import Client

mindee_client = Client(passport_token="your_passport_token_here")

passport_data = mindee_client.parse_passport("/path/to/file")

print(passport_data.passport.checklist)</code></pre>

&nbsp;

&nbsp;

This sample code should print something like:

&nbsp;

<pre>
<code class="language-bash">{
 'mrz_id_number_checksum': True, 
 'mrz_date_of_birth_checksum': True, 
 'mrz_expiration_date_checksum': True, 
 'mrz_personal_number_checksum': True, 
 'mrz_last_name_checksum': True, 
 'mrz_checksum': True
}</code></pre>

&nbsp;

&nbsp;

The machine readable zone includes many different checksums, allowing us to validate that the data it contains is correct. You can find more details&nbsp;<a href="https://en.wikipedia.org/wiki/Machine-readable_passport">here</a>.

&nbsp;

mrz_id_number_checksum: ID number in mrz checksum validation

 
mrz_date_of_birth_checksum: Date of birth in mrz checksum validation

 
mrz_expiration_date_checksum:&nbsp;Expiry date&nbsp;in mrz checksum validation

 
mrz_personal_number_checksum:&nbsp;Personnal number in mrz checksum validation

 
mrz_last_name_checksum: Last name&nbsp;in mrz checksum validation

 
mrz_checksum:&nbsp;Full mrz checksum validation

&nbsp;

Here is an&nbsp;python&nbsp;code&nbsp;for validating machine readable zone checksums:

<pre>
<code class="language-python">def check_sum(s):
 """
 https://en.wikipedia.org/wiki/Machine-readable_passport
 :param s: string
 :return: checksum value for string s
 """
 assert type(s) is str
 
 checker = 0
 alpha_to_num = {c: 10 + i for i, c in enumerate('ABCDEFGHIJKLMNOPQRSTUVWXYZ')}
 for i, c in enumerate(s):
 if i % 3 == 0:
 weight = 7
 elif i % 3 == 1:
 weight = 3
 else:
 weight = 1

 if c == '&lt;':
 val = 0
 elif c.isalpha():
 val = alpha_to_num[c]
 else:
 val = int(c)
 checker += val * weight
 return str(checker % 10)</code></pre>

&nbsp;

Passport data validation using Mindee python helper library.

Passport data validation

<h1>Performances benchmark</h1>

&nbsp;

In a few steps, you can use the mindee python library to perform benchmarks of passport extraction perfomances against your test dataset.

&nbsp;

<ol>
	<li>Perform api calls for each file in your test set</li>
	<li>Create <code>mindee.Passport</code> objects from your data (csv, json or whatever)</li>
	<li>Perform the benchmark and analyze the results</li>
</ol>

&nbsp;

&nbsp;

&nbsp;

<h2>1. Perform api calls for each file in your test set</h2>

&nbsp;

Before running the benchmark, we need to collect all the passport predictions from Mindee API and store them so we can perform the benchmark later without calling the API again.

&nbsp;

To do so, assuming that all your files are stored into a &quot;./test_dataset&quot; folder and that we want to save all the responses into a &quot;./mindee_responses&quot; folder, here is the code:

&nbsp;

<pre>
<code class="language-python">from tqdm import tqdm
from mindee import Client
import os

TEST_DATA_PATH = "./passport_data"
RESPONSES_PATH = "./passport_responses"

mindee_client = Client(passport_token="your_passport_token_here")

# Loop over all your test files
for test_filename in tqdm(os.listdir(TEST_DATA_PATH)):
 # Get the current file path
 test_file_path = os.path.join(TEST_DATA_PATH, test_filename)

 # To make sure we don't stop the process if an error occurs
 try:
 # Parse the current file
 mindee_response = mindee_client.parse_passport(test_file_path)

 # Store the response inside json file to be restored later
 # In this example we use the test file name in the json filename to
 # be able to retrieve the corresponding file
 response_filepath = os.path.join(RESPONSES_PATH, test_filename + ".json")
 mindee_response.dump(response_filepath)
 except Exception as e:
 # If an error occurs, print the filename so you can understand
 # what happened
 print(test_filename, e)</code></pre>

&nbsp;

&nbsp;

&nbsp;

<h2>2. Create <code>mindee.Passport</code> objects from your data (csv, json,&nbsp;whatever...)</h2>

&nbsp;

The <code>mindee.Passport</code> class contains a <code>compare()</code> method that takes as inputs two <code>mindee.Passport</code> objects. Before running our final script, we need now to create a&nbsp;<code>mindee.Passport object</code> containing the true labels for each fields.

&nbsp;

We&#39;ll use a csv file in this example and the&nbsp;<a href="https://pandas.pydata.org/">pandas</a>&nbsp;library.

&nbsp;

<h3>ground_truth.csv header</h3>

<pre>
<code class="language-bash">,filename,birth_date,birth_place,country,expiry_date,gender,given_names,id_number,issuance_date,mrz1,mrz2,surname
</code></pre>

<h3>&nbsp;</h3>

To construct a Passport&nbsp;object from this dummy csv example, you can simply do:

&nbsp;

<pre>
<code class="language-python">import pandas as pd
from mindee import Passport

ground_truth_df = pd.read_csv("./labels.csv")


def passport_from_csv_row(df_row):
 # For variable lengths features, split the string to get a list
 given_names = df_row["given_names"].split("|")
 return Passport(
 birth_date=df_row["birth_date"],
 birth_place=df_row["birth_place"],
 country=df_row["country"],
 expiry_date=df_row["expiry_date"],
 gender=df_row["gender"],
 given_names=given_names,
 id_number=df_row["id_number"],
 issuance_date=df_row["issuance_date"],
 mrz1=df_row["mrz1"],
 mrz2=df_row["mrz2"],
 surname=df_row["surname"]
 )


for index, df_row in ground_truth_df.iterrows():
 passport_truth = passport_from_csv_row(df_row)
 print(passport_truth)
</code></pre>

&nbsp;

&nbsp;

Running this code should print in your console something like this:

&nbsp;

<pre>
<code class="language-bash">-----Passport data-----
Filename: passport.png 
Full name: HENERT PUDARSAN 
Given names: HENERT 
Surname: PUDARSAN
Country: GBR
ID Number: 707797979
Issuance date: 2012-04-22
Birth date: 1995-05-20
Expiry date: 2017-04-22
MRZ 1: P&lt;GBRPUDARSAN&lt;&lt;HENERT&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;
MRZ 2: 7077979792GBR9505209M1704224&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;00
MRZ: P&lt;GBRPUDARSAN&lt;&lt;HENERT&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;7077979792GBR9505209M1704224&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;00
----------------------</code></pre>

&nbsp;

&nbsp;

&nbsp;

<h2>3.&nbsp;Perform the benchmark and see the results</h2>

&nbsp;

Last step, now we need to wrap it all up.

&nbsp;

The Benchmark class has two methods <code>Benchmark.add</code>&nbsp;and <code>Benchmark.save</code>&nbsp;for adding comparison between two Passport objects, and saving the final metrics.

&nbsp;

&nbsp;

<pre>
<code class="language-python">import pandas as pd
from mindee import Response, Passport, Benchmark
import os

RESPONSES_PATH = "./passport_responses"
BENCHMARK_PATH = "./benchmark_passport"

benchmark = Benchmark(BENCHMARK_PATH)
ground_truth_df = pd.read_csv("./labels.csv")


def passport_from_csv_row(df_row):
 given_names = df_row["given_names"].split("|")
 return Passport(
 birth_date=df_row["birth_date"],
 birth_place=df_row["birth_place"],
 country=df_row["country"],
 expiry_date=df_row["expiry_date"],
 gender=df_row["gender"],
 given_names=given_names,
 id_number=df_row["id_number"],
 issuance_date=df_row["issuance_date"],
 mrz1=df_row["mrz1"],
 mrz2=df_row["mrz2"],
 surname=df_row["surname"]
 )


# Loop over each file in our csv
for index, df_row in ground_truth_df.iterrows():
 try:
 # Create ground truth passport object
 ground_truth_passport = passport_from_csv_row(df_row)

 # Load the mindee Response for the current file
 mindee_response = Response.load(os.path.join(RESPONSES_PATH, df_row["filename"] + ".json"))

 # Add the comparison between the two passports to the benchmark
 benchmark.add(
 Passport.compare(mindee_response.passport, ground_truth=ground_truth_passport),
 df_row["filename"]
 )

 except Exception as e:
 print(df_row["filename"], e)

benchmark.save()</code></pre>

&nbsp;

&nbsp;

Inside our benchmark folder, you should see a new directory was created, and a metrics.png file shoud have been created inside with the different metrics:

&nbsp;

<img src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2020/11/03/metrics.png" />

&nbsp;

The Passport benchmark runs on the 12 fields as shown above. For each of them, you get an information of:

accuracy: the proportion of correct predictions

precision: The proportion of correct predictions among all the non null predictions

Performances benchmark framework using Mindee python helper library for Passport OCR APIs.

Parsing Passports

<h1>&nbsp;Extracting License Plate Details</h1>

&nbsp;

After a car accident, knowing all the information about the vehicles involved is critical for insurance purposes.&nbsp; You need to have the driver&#39;s information, and information about the car.&nbsp; Imagine if you could collect it all with just your camera, and upload the photos to your adjuster automatically.

&nbsp;

Alternatively, imagine you run a trucking yard, and want to track the arrival and departures of the trucks.&nbsp; By reading the license plates - you can accurately timestamp truck movements.

&nbsp;

But a photo of a license plate is just a photo - how do we extract the information from the plates for entry in to our processes?

&nbsp;

With Mindee&#39;s License Plate API, you just upload an image of a vehicle (or vehicles for multiple plates) and the plates are identified and the text of the plate extracted.

&nbsp;

<h2>API Prerequisites</h2>

&nbsp;

<ol>
	<li>You&rsquo;ll need a free Mindee account.&nbsp;<a href="https://platform.mindee.net/signup" target="_blank">Sign up</a>&nbsp;and confirm your email to login.</li>
	<li>A photo of a car or turck witha European license plate visible. (The API is good, but not magic).</li>
</ol>

&nbsp;

<h2>Setting Up the API</h2>

&nbsp;

Log into your Mindee account and select the European License Plate API:

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/10/08/screenshot-2020-10-08-at-210825.png" />

&nbsp;

To activate the API, click the &ldquo;Try for Free&rdquo; button to access our generous free tier. You&rsquo;ll land on the dashboard page - where you can quickly see API usage (you have none right now, but that will change).&nbsp; On the left navigation, there are links to &ldquo;Documentation&rdquo;, &ldquo;Credentials&rdquo; and &ldquo;Live Interface&rdquo;.&nbsp; The docs tab has all of the technical details you&rsquo;ll need to build for the receipts API endpoint, and the Live Interface is a cool interactive demo. Rather than try out the demo, we want to build with the API,&nbsp; so click on &ldquo;Credentials&rdquo; to create an API token.

&nbsp;

Add a new token. In this example, I&rsquo;ve named it &ldquo;Tutorial&rdquo;:

&nbsp;

<img src="https://mindee-public-website.s3.amazonaws.com/blog/2020/09/01/pasted-image-0-1.png" />

&nbsp;

Click &ldquo;Add New Key&rdquo; and you&rsquo;ll be able to see your API token,.

&nbsp;

<h2>Making Your First API Call</h2>

&nbsp;

We&#39;ll use curl from the terminal for your first call.&nbsp; If you visit the Documentation tab, you can copy the code with your API key and paste directly into your command line

&nbsp;

<pre>
<code class="language-bash">curl -X POST \
 https://api.mindee.net/products/license_plates/v1/predict \
 -H 'X-Inferuser-Token: {token}' \
 -H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
 -F file=@/Users/MyMac/Downloads/rewe_truck.jpg</code></pre>

&nbsp;

The image I am using is a truck parked in a lot:

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2020/10/08/rewe_truck.jpg" />

<h2>&nbsp;</h2>

<h2>The Response</h2>

&nbsp;

The API response is JSON:

<pre>
<code class="language-json">{
	"call": {
		"endpoint": {
			"name": "license_plates",
			"version": "1.0"
		},
		"finished_at": "2020-10-08T20:14:26+00:00",
		"id": "58d3293f-b97b-4f5a-85ad-ed64a386f02f",
		"n_documents": 1,
		"n_inputs": 1,
		"processing_time": 0.474,
		"started_at": "2020-10-08T20:14:25+00:00"
	},
	"documents": [
		{
			"id": "22d5c578-ca4f-4b79-8071-dbd510e6db3c",
			"name": "rewe_truck.jpg"
		}
	],
	"predictions": [
		{
			"license_plates": [
				{
					"probability": 0.97,
					"segmentation": {
						"bounding_box": [
							[
								0.427,
								0.808
							],
							[
								0.515,
								0.808
							],
							[
								0.515,
								0.841
							],
							[
								0.427,
								0.841
							]
						]
					},
					"value": "BI04176"
				}
			]
		}
	]
}</code></pre>

&nbsp;

In&nbsp;under half a second, the API returned a value for the license plate &quot;BI04176&quot; with 97% confidence.&nbsp; The bounding box lists the [x,y] coordinates forming a rectangle around the plate in the image.

&nbsp;

<h2>Summary</h2>

&nbsp;

In just a few lines of code - we are able to automatically parse and extract the&nbsp;license plate information from a photo.&nbsp; There are a number of applications that can now be implemented with a connected camera, the Mindee API and a little bit of code.

&nbsp;

Let us know how you are planning to use the License plate API in your application!

Many objects in our world have unique identifiers.  For cars, the most visible and commonly used identifier is the license plate.  Every vehicle on the road has one, and they are used by police to issue tickets, identify speeders, etc.  But can we harness this power for our own applications?

Using Mindee's license plate API we can obtain the License plate simply from a photo of a vehicle.

Extract License Plate Data

<h1>Invoice parsing automation Microsoft Flow: Mindee and Excel</h1>

&nbsp;

Mindee&rsquo;s invoice parsing API makes the extraction of invoice details quick and easy.&nbsp; Our RESTful API takes an image or PDF of an invoice and extracts all of the pertinent details of the invoice.&nbsp; In under a second, it returns a JSON response with the extracted elements along with coordinates of their visual location in the document and the confidence of our ML algorithm.&nbsp; We&rsquo;ve blogged on how to use the Invoice API with <a href="https://mindee.com/documentation/apis/invoice-parsing">Curl</a>,&nbsp;<a href="https://mindee.com/documentation/apis/invoice-parsing/invoice-parse-nodejs">NodeJS</a>&nbsp;and <a href="https://mindee.com/documentation/apis/invoice-parsing/python-invoice">Python</a>.

&nbsp;

We realise that this is just part of the invoice &lsquo;process&rsquo; at your company. In this post, we&rsquo;re going to extend the API further using Microsoft Power Automate.&nbsp; By creating a &lsquo;flow&rsquo; for your invoices you can build automation connecting different Office 365 services to automate your workflows.

&nbsp;

<h2>Robotic Process Automation (RPA)</h2>

RPA is using computers and automation to streamline mundane tasks. By reducing the amount of time employees have to do simple and repetitive tasks, the employyes have more time for high value work - increasing productivity and reducing human&nbsp;errors that might occur during the process.&nbsp; The process described below certainly removes a lot of process steps to the invoice process, so it is an RPA.

<h3>&nbsp;</h3>

<h3>Low Code/ No Code</h3>

&nbsp;

Even better, this process also can be considered a low code implementation.&nbsp; There&#39;s no need to pull one of your developers off a&nbsp;project to implement. Simply following these instructions (and having access to an Office 365 account), any member of staff should be able to add this automation step to the invoice process.

&nbsp;

<h2>Complete Invoice Processing</h2>

Extracting the pertinent data from the invoice is a huge step in simplifying the processing of an invoice, but it is just one step in the required workflow.&nbsp; In today&rsquo;s digital age, many invoices arrive via email, and must be rescued from your inbox for processing. Then all the details must be entered into a spreadsheet (or other database) - all this before you actually *PAY* the invoice.&nbsp;

&nbsp;

Many of these steps can be automated, and with&nbsp;<a href="https://mindee.com/documentation/apis/invoice-parsing">Mindee&rsquo;s Invoices API</a>, the most tedious part (the data entry into the payment system) can be automated as well. If your company is using Microsoft Office 365, this automation can be done using Microsoft&#39;s&nbsp;Power Automate flows.

&nbsp;

In the following example, we&rsquo;ll automate the process from email delivery to storing the invoice data into an Excel Online spreadsheet. Inside the workflow itself, we&rsquo;ll save the invoice to a document library SharePoint Online, call the Mindee&rsquo;s Invoices API and populate additional, custom columns in the SharePoint library and. A picture is worth a thousand words, so here is a diagram of the flow as we built it:

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-190441.png" style="height:708px; width:1200px" />

&nbsp;

<h3>Test the Flow</h3>

&nbsp;

You can try this flow by sending an email to <a href="mailto:doug@riolinx.com">doug@riolinx.com</a>&nbsp;(amke sure you attach an invoice in PDF or JPG&nbsp;format). Microsoft Flows seem to run every 5 minutes or so, so you&rsquo;ll soon see your invoice parsed and added to the <a href="https://riolinx365-my.sharepoint.com/:x:/g/personal/doug_riolinx_com/EcHR-jrlYSRBkFMx6DbI7mUBzawhzsBGc1CsDAo27fVD5Q?e=rpJRgx">Excel document</a>.&nbsp; Feel free to try it out as often as you&rsquo;d like!

While this is a fu example, it does not do a lot of good for your company&#39;s invocies to end up in a publically available spreadsheet on the internet.&nbsp; You&#39;ll want to create your own (private) version of this Flow for your company&#39;s invoices.&nbsp; Read on to learn how:

&nbsp;

<h2>Importing the Flow</h2>

&nbsp;

The flow we created is available on <a href="https://github.com/publicMindee/microsoft-flow-invoice-integration">Github</a>.&nbsp; Download the &quot;MindeeInvoiceswithMicrosoftFlow.zip&quot; from the repository.&nbsp; (You&#39;ll also need the invoice_excel.xlsx file in a bit, so you might as well download that now as well.).

To add the Flow to your Microsoft account,&nbsp;navigate to <a href="https://flow.microsoft.com">https://flow.microsoft.com</a>&nbsp;and log in with your work credentials.&nbsp; You may have to create a free trial of <a href="https://powerapps.microsoft.com/en-us/">Power Apps</a> if you do not already have Flow/Power apps enabled.&nbsp;

Click on the &ldquo;My Flows&rdquo; in the left navigation.&nbsp;&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-191556.png" style="height:470px; width:500px" />

&nbsp;

On&nbsp;this page, there is a link to import a flow:

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-191643.png" style="height:198px; width:500px" />

&nbsp;

Click import to begin the upload of the Zip file. After the Zip file is uploaded, the page&nbsp;will ask you to name the flow, and to connect it to your Outlook account. The connection is required in order for the Flow to have access to your emails.

&nbsp;

Step 1:&nbsp; Under name - import setup - Create as new, and give the Flow a name that makes sense for your organization.

 
Step 2: Connect your Outlook - click &ldquo;select during import.&rdquo;&nbsp; In my case, I did not have a Flow connection set to Outlook, so I had to click &ldquo;Create New&rdquo; and connect to my SharePoint account.&nbsp; Clicking the refresh allowed me to add the connection:

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-191854.png" style="height:215px; width:500px" />

&nbsp;

Here I am connecting to a new email adress in a new Microsoft Flow account.

Now, once all the connections have been made, you can press the blue import button at the bottom of the page.&nbsp;After a few seconds, the&nbsp;package will import successfully.

&nbsp;

<h2>Editing the flow</h2>

&nbsp;

In order for this flow to succeed in your Office 365 environment, you&rsquo;ll need to make a few edits. Head back to &ldquo;My Flows&rdquo; and select the Mindee integration and press the edit button.

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-192227.png" style="height:74px; width:500px" />

&nbsp;

<h3>Add your email account</h3>

&nbsp;

The email account that receives invoices in your organization. In the flow, it is currently set to <a href="mailto:doug@riolinx.com">doug@riolinx.com</a> - which does your organisation very little good.&nbsp; We&rsquo;ll need to update that:

Click the &ldquo;On new email&rdquo; and select advanced options:

 
<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-192558.png" style="height:130px; width:500px" />

&nbsp;

Now, edit the &ldquo;To&rdquo; field to correspond to the address that will be receiving invoices in your organization.

&nbsp;

&nbsp;

<h3>Mindee API Key</h3>

The next step in the process is your Mindee API Key.&nbsp; You&rsquo;ll need to paste in your API key in the Value field.&nbsp; To obtain a Mindee Invoice API Key, visit Mindee.com and follow these <a href="https://mindee.com/documentation/apis/invoice-parsing">steps to get your key</a>.

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-193946.png" style="height:198px; width:500px" />

&nbsp;

Now you can save your Flow. To turn it on, head back to the main view of your flow, and select &ldquo;Flow Checker&rdquo;. You;ll see a warning that the flow is off - and a button to turn it on.

&nbsp;

This will flag another warning - don&rsquo;t worry - it is just telling you that the flow has never run successfully.&nbsp; Of course it hasn&rsquo;t - you haven&rsquo;t had a chance yet!

&nbsp;

<h3>Test it out</h3>

&nbsp;

Send an invoice to the email address you configured in the flow (don&#39;t forget to attach&nbsp;an invoice!).&nbsp; The Flow will run in the next few minutes, and you&rsquo;ll see the successful run in your 28 day run history.

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-194218.png" style="height:182px; width:500px" />

&nbsp;

Clicking the date and time (in the 28 day run history table) allows you to check the status. If you scroll to the bottom of the Flow, you&rsquo;ll see the&nbsp; &ldquo;apply to each attachment in the email.&rdquo; Open this, and then Open &ldquo;Update Excel Online Spreadsheet&rdquo;, and the open one of the variables:

 
<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-194415.png" style="height:478px; width:500px" />

&nbsp;

The screenshot above shows that the APi correctly extracted the invoice date as 29 June 2020.&nbsp; This is super cool - as we now have a Flow that interacts with the Mindee API to extract data from submitted invoices.

&nbsp;

But your Flow does not actually DO anything with that data yet.&nbsp; In the next sections, we&rsquo;ll connect the flow to Excel and to SharePoint, and submit the data to these Office tools for future manipulation.

&nbsp;

<h2>Connect an Excel Online Spreadsheet&nbsp;&nbsp;</h2>

&nbsp;

In the Github repo, there is an excel file: Invoice_excel.&nbsp; Upload this into your Excel Online account.&nbsp; To do this: Open Excel in a new tab:

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-200759.png" style="height:249px; width:500px" />

&nbsp;

Upload the file into Excel. It will open, and you&rsquo;ll find an empty table with headers that llok like they will intake invoice data. This table is named &ldquo;invoices2021&rdquo;

&nbsp;

<h3>Connect the Flow to the Excel table:</h3>

&nbsp;

Back in your Flow, click the edit button and open the &ldquo;apply to each Attachment in the email&rdquo; (at the bottom).&nbsp; Open the &ldquo;Update Excel spreadsheet&rdquo; entry and add an action at the bottom(the circle with a &#39;+&#39; sign in it).&nbsp; We&rsquo;re going to &ldquo;add a row to a table&rdquo;:

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-200952.png" style="height:408px; width:500px" />

&nbsp;

Choose the Invoice_excel file you just uploaded:

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-201215.png" style="height:193px; width:500px" />

&nbsp;

And the table is &ldquo;invoices2021.&rdquo;&nbsp; Now that this is added, all of the columns in the table will appear in the Flow.&nbsp; Add the variables that were assigned earlier by clicking each entry&nbsp;box in the form, and adding the dynamic variables as shown below (the purple dynamic entries correspond to the variables that exit the Mindee flow):

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-201314.png" style="height:396px; width:500px" />

&nbsp;

Click save, and test the flow. You can use the trigger you used in the initial test to run the flow immediately.

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-201406.png" style="height:444px; width:500px" />

The difference is now that the invoice details are added automatically to the Excel file.

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-201520.png" style="height:57px; width:500px" />

<h2>Yes!!!</h2>

<h2>&nbsp;</h2>

<h2>Saving the file to SharePoint</h2>

In My SharePoint site, I have created a new document library called &quot;invoice&quot;. When it is created, I land on a&nbsp;page with a table where my files will end up.

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-214645.png" style="height:154px; width:500px" />

&nbsp;

Looking at the column names, I see that I can add columns to my table!&nbsp; We can add the extracted Mindee data right into the SharePoint Table:

&nbsp;

Clicking &quot;add column&quot; (note: always choose &quot;single line of text&quot; - in case the API reponds with an &quot;N/A&quot;)&nbsp;opens this dialogue:

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-215105.png" style="height:845px; width:500px" />

&nbsp;

For this example, I picked 4 variables: supplier name, invoice date, due date, and total due. I could have added more - it is up to you what data you want to appear here.

&nbsp;

<h2>Connect the Flow to SharePoint</h2>

Now that we&#39;ve created the livrary, we need to wire it to connect to our automated Flow.

First we&#39;ll want to insert the file into the document library.&nbsp; So between the last variable, and before we add to the Excel table, insert a step Sharepoint: Create File:

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-220039.png" style="height:272px; width:500px" />

&nbsp;

Fill in the form - the SharePoint and Folder path are your companiy&#39;s SharePoint, and the flibrary you just created for your invoices.&nbsp; Then use the dynamic variables to add the &quot;Attachment Name&quot; and &quot;attachment contents&quot; to the last two fields.

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-220131.png" style="height:181px; width:500px" />

&nbsp;

The file is now added, but to fill the new columns created in the library, we need yet another ocnnector:&nbsp; SharePoint: Update File Properties:

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-220811.png" style="height:393px; width:500px" />

&nbsp;

We again connect this to our new SharePoint Library.&nbsp; To specifiy which line we apply, choose ItemId - this refers back to the document we just created one step back.&nbsp; When you add the library name, it will find all of the columns, you added. Then we can use the variables from the Mindee Flow to fill out the columns of the SharePoint library, as shown above.

&nbsp;

<h2>Adding the SharePoint Path to Excel</h2>

&nbsp;

The Excel table has fields to help you find the file in SharePoint, so go back to the &quot;add a row to excel: and insert the SharePoint Filename and path to the flow, and connect the dots between all the files and connections we&#39;ve made with this flow.

&nbsp;

Now we&#39;re ready to save the whole flow, and test!

&nbsp;

<h2>The test</h2>

&nbsp;

Once we send an email to the account, we see that not only is the invoice added to the SharePoint Library, but the details are also added to the excel table (with the SharePonit file details in the Excel file):

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-222724.png" style="height:57px; width:500px" />

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/13/screenshot-2021-01-13-at-222714.png" style="height:40px; width:500px" />

&nbsp;

<h2>Conclusion</h2>

&nbsp;

To increase productivty, many organisations are using Robotic Process Automation and low/no-code tools to automate tedious and slow tasks.&nbsp; The processing of invoices is one such process that is readily suited for automation.&nbsp;

Using Mindee&#39;s API to extract&nbsp;data from an invoice is just one piece of the invoice pipeline. In this demo, we&#39;ve extended the use of the Mindee invoice extraction API with Microsoft Flow - taking an email attachment, saving it in SharePoint, and extracting the details as column data in SharepPoint as well as creating a row in Excel.

With all of the tools inside Microsoft 365, this is just the tip of the iceberg - you could potentially integrate with any other product in the suite. Try out our Mindee Flow to extract data, and then use the variables to wire the data into your favorite Microsoft Project.&nbsp; If you have any questions (how to reach out, etc.)

&nbsp;

&nbsp;

&nbsp;

Extracting the details of an invoice is often just a small part of the process flow that each organisation takes.  In this post, we'll walk you through the steps on how to use Mindee's Invoice extraction software to populate a Microsoft Excel table, and a SharePoint repository - creating a full invoice processing pipeline.

Invoice parsing automation Microsoft Flow: Mindee and Excel

Tutorials

<h1>How to design and deploy your first document parsing API</h1>

&nbsp;

&nbsp;

<iframe frameborder="0" height="500px" src="https://www.youtube.com/embed/uz5YV4RknH4" width="100%"></iframe>

<h2>&nbsp;</h2>

<h2>&nbsp;</h2>

<h2>Prerequisites</h2>

&nbsp;

<ol>
	<li style="list-style-type:decimal">You will need a free beta account.<a href="https://beta.mindee.net" style="text-decoration:none"> Sign up</a> and confirm your email address before proceeding.</li>
	<li style="list-style-type:decimal">Download this <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/13/burger-stores_config.json" style="text-decoration:none" target="_blank">data model configuration file</a>.</li>
	<li>Download this <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/13/burger-stores.zip" style="text-decoration:none" target="_blank">sample training set</a>.</li>
</ol>

&nbsp;

&nbsp;

This tutorial describes the steps required to create a custom document data extraction API using Mindee.

&nbsp;

Over the course of this tutorial, you will learn how to train a deep learning model that can parse and extract the data of your choice in your documents.

&nbsp;

Once the ML model is trained, you will be able to use the custom REST API backed by that model in your application code using the language of your choice.

&nbsp;

Let&rsquo;s see how to build and deploy your first document data extraction REST API using the following, fictitious scenario: as a developer, you want to extract the name of any burger food store that delivers orders to your door, along with its address, average user rating, user votes count as well as its first menu item name and price.

&nbsp;

&nbsp;

<h2>Set up your API</h2>

&nbsp;

Once you have signed up and logged in at <a href="https://beta.mindee.net" style="text-decoration:none"> https://beta.mindee.net</a>, press the I&rsquo;m ready, start building button, as shown below:

&nbsp;

&nbsp;

<img src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/14/apibuilder-apihub-homepage.png" />

&nbsp;

&nbsp;

First, we need some basic information about your new, custom document scanning API. That info will help us highlight your API in your Mindee API Hub (and in other users&rsquo; API Hubs if you choose to share your API with them). The requested properties are:

&nbsp;

&nbsp;

<ul>
	<li>Document type: The common name of the document type your API will be trained on. 
	Examples: 1040 US Tax Form, Pay Stub, Bank Statement </li>
	<li>API name: The resource name of your API URL. 
	We automatically generate one for you based on your document type but you can customize it at your own convenience. Note however that you can&rsquo;t use special or accented characters. The format of your API URL complies with the following scheme: <code>https://api-beta.mindee.net/[username]/[API_name]</code></li>
	<li>Description: An optional, short blurb aimed at conveying the purpose of the API. This description will appear below the image in your API Hub (and others if you choose to share your API with other users). We suggest you make it as clear and concise as possible to maximize its effectiveness.</li>
	<li>Image: An uploaded image that appropriately illustrates your API. Other users may see it in their API Hub if you choose to share it with them.</li>
</ul>

&nbsp;

&nbsp;

Here is a possible configuration for our first burger stores API:

&nbsp;

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/14/api-builder-api-setup.png" />

&nbsp;

&nbsp;

Fill out the form with the following values (or pick your own):

&nbsp;

<ul>
	<li style="list-style-type:disc">Document type: Burger Stores and Menus</li>
	<li style="list-style-type:disc">API Name: burger_stores</li>
	<li style="list-style-type:disc">Description: An API used to extract data (such as store name, ratings and address) from popular burger stores</li>
	<li style="list-style-type:disc">Image: upload <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/13/burger_api.jpg" style="text-decoration:none" target="_blank">this image</a></li>
</ul>

&nbsp;

Once you&rsquo;re done setting up your API, press the Next step: define your data model button to create the list of fields we&rsquo;d like to extract from our burger stores documents.

&nbsp;

&nbsp;

&nbsp;

<h2>Define your document data model</h2>

&nbsp;

&nbsp;

The following page shows up:

&nbsp;

 
<img alt="" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/14/api-builder-design-empty.png" width:1792px="" />

&nbsp;

&nbsp;

You can either manually add each field or upload a json-formatted file, typically generated from an existing data model someone previously built (and exported).

&nbsp;

The following information is required for each field:

&nbsp;

<ul>
	<li style="list-style-type:disc">Field name: The legible field name used in your API documentation</li>
	<li style="list-style-type:disc">Api response key: The field key available in your API response to carry the extracted field value for each document</li>
	<li style="list-style-type:disc">Field type: The data type of the field you are trying to extract. 
	The supported data types currently are: strings, numbers, dates, email addresses, phone numbers and URLs. 
	Specifying a field type helps our deep learning selector better understand the data you&rsquo;re looking for by restricting the number of potential candidates for each field. <a href="https://mindee.com/documentation/api-builder/data-model" style="text-decoration:none">Learn more about the data types</a> we support.</li>
</ul>

&nbsp;

For the purpose of this tutorial, we want to extract the following information from our image or PDF documents:

&nbsp;

<ul>
	<li style="list-style-type:disc">The burger store name (a string)</li>
	<li style="list-style-type:disc">The burger store address (a string)</li>
	<li style="list-style-type:disc">The store average user rating (a floating number)</li>
	<li style="list-style-type:disc">The votes count number (an integer)</li>
	<li style="list-style-type:disc">The name of the first item on the menu (a string)</li>
	<li style="list-style-type:disc">The price of the first item (an amount)</li>
</ul>

&nbsp;

Below is a screenshot of one of our training files with each field above highlighted in green:

&nbsp;

&nbsp;

&nbsp;

<img src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/14/api-builder-fields.png" />

&nbsp;

&nbsp;

&nbsp;

The Mindee representation of these fields after setting them up one by one is the following:

&nbsp;

&nbsp;

&nbsp;

<img alt="" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/14/api-builder-design-burger-stores-model.png" width:1202px="" />

 
&nbsp;

You can manually add them one by one or you can upload a configuration file to automatically populate your data model (this file is typically generated using the Config Export feature available to existing models in the Settings section of your API dashboard).

&nbsp;

Press the Choose file button to upload the <a href="http://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/13/burger-stores_config.json" style="text-decoration:none" target="_blank">json-formatted config file</a> (the one mentioned in the Prerequisites section above):

&nbsp;

&nbsp;

<img alt="" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/14/api-builder-design-choose-file.png" width:579px="" />

&nbsp;

Once you&rsquo;re done setting up your data model, press the Data model ready? start training button at the top of the screen:

&nbsp;

 
<img alt="" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/14/api-builder-design-ready.png" width:1167px="" />

&nbsp;

<h2>&nbsp;</h2>

<h2>&nbsp;</h2>

<h2>Train the model</h2>

&nbsp;

Now is the time to train our Burger Stores deep learning model in the Training section of our API:

&nbsp;

&nbsp;

<img alt="" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/14/api-builder-training-empty.png" width:1792px="" />

&nbsp;

&nbsp;

For each of our documents we must assign a value to each field by selecting a box (or several consecutive boxes) displayed in the uploaded document. The Mindee selector highlights all the potential candidates for each field in blue, as shown below:

&nbsp;

&nbsp;

 
<img alt="" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/14/api-builder-training-burger-stores-onedoc.png" width:1792px="" />

&nbsp;

&nbsp;

When you find the blue box that matches the field you are looking for, simply click that box. This populates the field text box with the selected value and shows the selected box in an identifiable color. For instance, the screenshot below highlights the box we selected for the average_rating field in orange:

&nbsp;

&nbsp;

<img alt="" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/14/api-builder-training-selected-box.png" width:1792px="" />

&nbsp;

&nbsp;

If the content you are looking for is composed of more than one box, simply select all the consecutive boxes that match your field (non-consecutive boxes may result in an inefficient model). For example, he screenshot below shows the address field populated with 9 consecutive boxes:

&nbsp;

&nbsp;

 
<img alt="" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/14/api-builder-training-multiple-boxes.png" width:1254px="" />

&nbsp;

&nbsp;

You can<a href="http://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/13/burger-stores.zip" style="text-decoration:none" target="_blank"> download this sample dataset</a> to train your Burger Stores deep learning model. Unzip the file into a folder and upload each file in that folder one-by-one to the Training page.

&nbsp;

Once you have selected the proper box(es) for each of your fields (as displayed on the right-hand side of the document), press the validate button and repeat the tagging process for the next document.

&nbsp;

The progress bar above each picture indicates the advancement of your tagging efforts and identifies the milestones at which a new model is generated.

&nbsp;

Every 20 documents, a new model is being trained and automatically deployed into your API environment. You will receive an email whenever your latest model has been trained and is up and running:

&nbsp;

&nbsp;

<img alt="" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/14/api-builder-training-model-ready-email.png" width:670px="" />

&nbsp;

&nbsp;

&nbsp;

Note that the more documents you feed into your model, the more accurate it will be.

&nbsp;

With that being said, we will generate the first version of your trained model once you reach the 20 documents threshold (uploaded and validated).

&nbsp;

At this point, you can experience the true power of the Mindee data extraction engine every time you upload a new training file, as the model will start filling out the fields on the right-hand side with the most likely values it was able to identify in the uploaded file.

&nbsp;

From now on, the Burger Stores model will help you annotate new data in the Training section:

&nbsp;

&nbsp;

<img alt="" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/14/api-builder-training-model-predictions.gif" width:1222px="" />

&nbsp;

&nbsp;

With a 20-documents training set, the model algorithm starts working pretty well on most fields, but still makes a few mistakes on the complex fields, such as the address field (extra words are predicted).

&nbsp;

Once again, the more data you&rsquo;re feeding to the model, the more accurate your API will get. So your job will now consists in correcting your model until you reach the next 20 documents threshold, which will kick off the generation of a new, more efficient model. You may repeat this task as often as you want.

 
&nbsp;

&nbsp;

<h2>Settings, API keys and documentation</h2>

&nbsp;

If you select the Documentation tab, the following page is displayed: 

&nbsp;

&nbsp;

<img src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/14/api-builder-documentation.png" />

&nbsp;

&nbsp;

This page allows you to explore your API endpoints and download your API definition as an <a href="https://www.openapis.org/" target="_blank">OpenAPI</a>-formatted file.

&nbsp;

The API Keys tab allows you to create and manage your custom API tokens:

&nbsp;

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/14/api-builder-apikeys.png" />

&nbsp;

&nbsp;

The Settings tab allows you to update general information about your API such as its name or its description. It also allows you to download your data model field configuration, which can be used to create a new, enhanced data model without having to start from scratch.

&nbsp;

&nbsp;

<img alt="" src="https://mindee-public-website-staging.s3.amazonaws.com/blog/2021/01/14/api-builder-settings.png" />

 
&nbsp;

That&rsquo;s it! You should now be able to set up a real world use case now and build whichever document parsing API you may need.

&nbsp;

If you have any questions or feedback about this tutorial, feel free to reach out to us in the #tutorials channel of the <a href="https://mindeecommunity.slack.com" style="text-decoration:none" target="_blank">Mindee Community on Slack</a> (the Get Started email you have received when signing up contains an invitation to join it).

Build and deploy your own document parsing API using our deep learning engine.

<h1>&nbsp;Define your custom Data Model</h1>

&nbsp;

&nbsp;

In order to train your custom parsing API, you need to define a data model, i.e. a list of fields and their corresponding data types you want to extract from your documents.

&nbsp;

This tutorial will walk you through the steps of defining such a Data Model.

&nbsp;

For further information on specific data types, see arborescence on the left.

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net">Sign up</a> and confirm your email to login.</li>
</ol>

&nbsp;

&nbsp;

Let&rsquo;s get started!&nbsp;

&nbsp;

&nbsp;

After giving a name, description and cover image to your new API, you should land on the following page

&nbsp;

<img src="https://lh3.googleusercontent.com/DyZx4svCWQ8RN393xkXsc2F7CVzMLKYP2Un3H5k5o6iTEmb4vP27WpJ1F29Y9RUT-56EKEaTjTr2RTx5PiXaZVj6-vlxUTcndQY7lnne9Sq1idc8wJ3o_Mj6nkKIfD15ryinvgwQ" />

&nbsp;

From there you have two options:

&nbsp;

1- Manually add fields one by one by filling the form on the right side of the screen

2- Upload a Data Model config file

&nbsp;

Let&rsquo;s start with the manual option.

&nbsp;

&nbsp;

<h2>Manually add a field to your Data Model</h2>

&nbsp;

You can add a new field by filling in the right-side form with the following information:

&nbsp;

Field Name: The straightforward name that will appear on the annotation interface later on. Use a name that means something to you when reading it

&nbsp;

API response key: The name of the key used in the API response scheme

&nbsp;

Field type: The Field Type specifies the type of information we are going to look for on the document and defines the data type that will be returned in the API response.

&nbsp;

You can choose among a drop-down list of pre-built data types between :

<ul>
	<li><a href="http://mindee.com">String</a></li>
	<li><a href="http://mindee.com">Number</a></li>
	<li><a href="http://mindee.com">Date</a></li>
	<li><a href="http://mindee.com">Email address</a></li>
	<li><a href="http://mindee.com">Phone number</a></li>
	<li><a href="http://mindee.com">Url</a></li>
</ul>

&nbsp;

&nbsp;

<h2>Update or Delete a field</h2>

&nbsp;

Now that you&rsquo;ve created you first field, you should see something like this

&nbsp;

<img src="https://lh3.googleusercontent.com/zsvreL0pLGuzroeU91XKYyd8PjFvbbCfUoqfS1DdrAo8ytQP7jj9veXSRLk2v3TAbIPuGEo-e5pCipQ6d0UEQBwfZpXjs9wWoSrEvUttthYSk8tv-fkIs-rwmb4U35dTA3lN-5Se" />

&nbsp;

From there you can:&nbsp;

&nbsp;

<ol>
	<li>Add a field : Manually add a new field to your Data Model following the same process. You can repeat this step as many times as you want, there is no limit is the number of fields you can extract from a document.</li>
	<li>Edit a field : Edit a specific field. You can change its Field name, API response key and Field type</li>
	<li>Delete a field : Delete a specific field from your data model</li>
	<li>Start training : When your Data Model is ready. Click here to automatically deploy your API and start training it. (Hyperlink :Go to the training tuto)</li>
</ol>

&nbsp;

&nbsp;

<h2>Upload a Data Model file config&nbsp;&nbsp;</h2>

&nbsp;

Alternatively, you can create a whole set of fields at once by uploading a config json file in the left-side section.&nbsp;

&nbsp;

<img src="https://lh3.googleusercontent.com/Dr8_OcYp-AtheuIyXTd2WVzfMIojgyBsCQTt_JwIkMkzhVBMoxI4RJr9wd2ngIpFk6AqU5f1DrI95y17gTHfEyQbCCG31iHrNqS5AUi7BdwRyGyimviQDejDT8RfANkTbCC4QoLT" />

&nbsp;

&nbsp;

For instance, with our started&nbsp;example, the config json file looks like this

&nbsp;

&nbsp;

<pre>
<code class="language-json">[
 {
 "cfg": {
 "alpha": -1,
 "numeric": 0,
 "lowercase": -1,
 "uppercase": -1,
 "max_length": 999,
 "min_length": 1,
 "capitalized": -1,
 "specialchars": -1
 },
 "name": "name",
 "semantics": "word",
 "public_name": "name"
 },
 {
 "cfg": {
 "is_integer": -1
 },
 "name": "rating",
 "semantics": "amount",
 "public_name": "rating"
 },
 {
 "cfg": {
 "alpha": -1,
 "numeric": 0,
 "lowercase": -1,
 "uppercase": -1,
 "max_length": 999,
 "min_length": 1,
 "capitalized": -1,
 "specialchars": -1
 },
 "name": "first_burger",
 "semantics": "word",
 "public_name": "first burger"
 },
 {
 "cfg": {
 "is_integer": -1
 },
 "name": "first_burger_price",
 "semantics": "amount",
 "public_name": "first burger price"
 },
 {
 "cfg": {
 "is_integer": 1
 },
 "name": "number_of_vote",
 "semantics": "amount",
 "public_name": "number of votes"
 },
 {
 "cfg": {
 "alpha": -1,
 "numeric": -1,
 "lowercase": -1,
 "uppercase": -1,
 "max_length": 999,
 "min_length": 1,
 "capitalized": -1,
 "specialchars": -1
 },
 "name": "address",
 "semantics": "word",
 "public_name": "address"
 }
]</code></pre>

&nbsp;

Each field is introduce by a cfg key and has the following attributes:

&nbsp;

name : the API response key. Mandatory.

semantics : the Field Type. Mandatory.

public_name : the Field Name. Mandatory.

cfg : An object of additional data constraints depending on the Field Type. Mandatory.

&nbsp;

&nbsp;

The possible parameters for the cfg object are all mandatory and are the following:

&nbsp;

semantics = word (String):

alpha : Does it contain alpha characters. -1 = sometimes, 0 = never, 1 = always. Default value = -1

numeric : Does it contain numeric characters. -1 = sometimes, 0 = never, 1 = always. Default value = -1

lowercase : Does it contain lowercase characters. -1 = sometimes, 0 = never, 1 = always. Default value = -1

uppercase : Does it contain uppercase characters. -1 = sometimes, 0 = never, 1 = always. Default value = -1

capitalized : Is the first letter capitalized. -1 = sometimes, 0 = never, 1 = always. Default value = -1

specialchars : Does it contain special characters. -1 = sometimes, 0 = never, 1 = always. Default value = -1

min_legnth : What is the minimum length of the word. Accepts INT. Default value = 1

max_length : What is the maximum length of the word. Accepts INT. Default value = 999

&nbsp;

semantics = amount :

is_integer : Is it an INT. -1 = sometimes, 1 = always. Default value = -1

Define the list of fields you want to extract from your documents.

<h1>Set up a string field</h1>

&nbsp;

&nbsp;

You want to extract&nbsp;string fields from your documents (full name, addresses, place..) ?

&nbsp;

You can do that with the API Builder by creating a string field in the Data Model configuration step. <a href="http://mindee.com">See Data Model configuration tutorial</a>.

&nbsp;

This section will walk you through the implications of setting up a string field and the different parameters you can specify.

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

&nbsp;

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net">Sign up</a> and confirm your email to login.</li>
</ol>

&nbsp;

&nbsp;

&nbsp;Let&rsquo;s get started!&nbsp;

&nbsp;

After giving a name, description and cover image to your new API, you should land on the following page:

&nbsp;

&nbsp;

<img src="https://lh3.googleusercontent.com/JAH2atjiTH8Gn2DzJd8LvTrh__SgBEWBE8WMLIAHTGEuz4wnsQhnTcZUYlpBUsjEyeDOYznOWadszIJjGsKCOsJN5I5Bl-YqbxjnI3s47XpTYNdk-ASt2qswK_y1fEeZeXtgK_T1" />

&nbsp;

&nbsp;

Build a string field by filling in the right-side form and selecting &ldquo;String&rdquo; in the Field Type drop-down menu.

&nbsp;

&nbsp;

<h2>Additional parameters</h2>

<h2>&nbsp;</h2>

If you have additional context about this specific field and how the information is usually displayed in your documents, you can specify it during the field set up.

&nbsp;

&nbsp;

<h3>Feed parameters through UI</h3>

&nbsp;

Through the UI, you may check two different checkboxes:

<ul>
	<li>It never contains any alpha characters. By checking this, the engine will never be able to predict a word or a list of words containing alpha characters as the correct value for this field.</li>
	<li>It never contains any numeric characters. By checking this, the engine will never be able to predict a word or a list of words containing numeric characters as the correct value for this field.</li>
</ul>

 
&nbsp;

<h3>Feed parameters through config.json file</h3>

&nbsp;

You may want to create a String field through by uploading a config.json file. <a href="http://mindee.com">See Data Model tutorial for more info</a>

&nbsp;

Doing this, you have access to more possible parameters for this String field than the two checkboxes on the UI. To do that, fill in the cfg objects in your config.json file with the following parameters:

&nbsp;

alpha : Does it contain alpha characters

possible values: -1 means sometimes;&nbsp; 0 means never; 1 means always.

default value: -1

&nbsp;

numeric : Does it contain numeric characters

possible values: -1 means sometimes;&nbsp; 0 means never; 1 means always.

default value: -1

&nbsp;

lowercase : Does it contain lowercase characters.

possible values: -1 means sometimes;&nbsp; 0 means never; 1 means always.

default value: -1

&nbsp;

uppercase : Does it contain uppercase characters.

possible values: -1 means sometimes;&nbsp; 0 means never; 1 means always.

default value: -1

&nbsp;

capitalized : Is the first letter capitalized.

possible values: -1 means sometimes;&nbsp; 0 means never; 1 means always.

default value: -1

&nbsp;

specialchars : Does it contain special characters.

possible values: -1 means sometimes;&nbsp; 0 means never; 1 means always.

default value: -1

&nbsp;

min_legnth : What is the minimum length of the word.

possible values: any INT.

default value: 1

&nbsp;

max_length : What is the maximum length of the word.

possible values: Any INT.

default value: 999

&nbsp;

A valid entry for a String field in a config.json should look like this:

<pre>
<code class="language-json">{
 "cfg": {
 "alpha": -1,
 "numeric": 0,
 "lowercase": -1,
 "uppercase": -1,
 "max_length": 999,
 "min_length": 1,
 "capitalized": -1,
 "specialchars": -1
 },
 "name": "name",
 "semantics": "word",
 "public_name": "name"
}</code></pre>

&nbsp;

<h2>&nbsp;</h2>

<h2>Impact for training and making predictions</h2>

&nbsp;

By default, a String field allows every information read on the document to be considered as an eligible candidate.

&nbsp;

Specifying the different parameters will have consequences for you in the training phase and when making predictions later on.

&nbsp;

Basically, it provides the engine with a list of criteria that a word or a list of words should match in order to be considered as a valid candidate or an eligible possible correct answer.

&nbsp;

Warning: This means that, for this specific field, we will never be able to extract any information in your document that does not match these criteria.

&nbsp;

Don&rsquo;t worry, the String data type is the less filtering one and the default parameters allow everything.

&nbsp;

This field type should be your go-to data type if you do not have sufficient context about your documents or if the information you are looking for is displayed in inconsistent ways between different documents.

Extract String fields from your document.

String

<h1>&nbsp;Set up a number field</h1>

&nbsp;

You want to extract either an amount, a float number or a list of numeric characters from your documents?

&nbsp;

You can do that with the API Builder by creating a number field in the Data Model configuration step. (See Data Model configuration tutorial).

&nbsp;

This section will walk you through the implications of setting up a number field and the different parameters you can specify.

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net">Sign up</a> and confirm your email to login.</li>
</ol>

&nbsp;

Let&rsquo;s get started!&nbsp;

&nbsp;

After giving a name, description and cover image to your new API, you should land on the following page

&nbsp;

<img src="https://lh6.googleusercontent.com/2pkLMdxUgnqnXmL5t0TKV6wiI1SCaS0WvsIskfI__mAm-vx8wFE_I_KCQ3FgPPMhcz-yARimYuB3wxxVVEBSfLxOgX4vofqm-GK5vspQSSbLv2dx4z54NdQo8NO0YbMYrrzNdF97" />

&nbsp;

&nbsp;

Build a number field by filling in the right-side form and selecting &ldquo;Number : Integer, float, amount&rdquo; in the Field Type drop-down menu.

&nbsp;

&nbsp;

<h2>Additional parameters</h2>

<h2>&nbsp;</h2>

If you have additional context about this specific field or how the information is usually displayed in your documents, you can specify it during the field set up.

&nbsp;

&nbsp;

<h3>Feed parameters through UI</h3>

&nbsp;

Through the UI, you may enable a checkbox:

&nbsp;

<ul>
	<li>It is always an integer. By checking this, the engine will never be able to predict as a correct value a number that cannot be parsed as an integer</li>
</ul>

<h3>&nbsp;</h3>

<h3>&nbsp;</h3>

<h3>Feed parameters through config.json file</h3>

&nbsp;

You may want to create a number field through by uploading a config.json file. (See Data Model tutorial for more info)

&nbsp;

You may fill in the cfg objects in your config.json file with the following parameter:

&nbsp;

is_integer : Is the number or amount an integer

possible values: -1 means sometimes; 1 means always

default value: -1

&nbsp;

A valid entry for a number field in a config.json should look like this:

&nbsp;

<pre>
<code class="language-json">{
 "cfg": {
 "is_integer": -1
 },
 "name": "rating",
 "semantics": "amount",
 "public_name": "rating"
}</code></pre>

<h2>&nbsp;</h2>

&nbsp;

<h2>Impact for training and making predictions</h2>

&nbsp;

<h3>Candidates</h3>

&nbsp;

By specifying the data type as Number, you will force the engine to only consider words and series of words that can be interpreted as numbers or amounts as eligible candidates.

&nbsp;

Warning: This means that, for this specific field, we will never be able to extract any information in your document that cannot be interpreted as an amount or a number.

&nbsp;

&nbsp;

<h3>Text parsed</h3>

&nbsp;

Anything that can be read an amount or a number.

&nbsp;

Ex: &ldquo;1,000,876.93&rdquo;, &ldquo;12&rdquo; and &ldquo;$98.78&rdquo; are Numbers

&nbsp;

Amounts or numbers written with alpha characters are not parsed as Numbers.

&nbsp;

Ex: &ldquo;fifteen&rdquo; or &ldquo;two hundred and twelve&rdquo; are not parsed as Numbers.

&nbsp;

If the information you are looking for is sometimes displayed as such, you should use a String field.

&nbsp;

&nbsp;

&nbsp;

Extrat numbers (amounts, floats, integer...) from documents.

Numbers (amount, float, integer..)

<h1>Set up a date field</h1>

&nbsp;

You want to extract a date from your documents?

&nbsp;

You can do that with the API Builder by creating a Date field in the Data Model configuration step. (See Data Model configuration tutorial).

&nbsp;

This section will walk you through the implications of setting up a Date field.

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

&nbsp;

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net">Sign up</a> and confirm your email to login.</li>
</ol>

&nbsp;

&nbsp;

Let&rsquo;s get started!&nbsp;

&nbsp;

&nbsp;

After giving a name, description and cover image to your new API, you should land on the following page

&nbsp;

<img src="https://lh5.googleusercontent.com/tKQpJs5ZAT_kL7Sek6y6yJrkdtXxiz3cIRV7ewRGvWc2AKJjqsAGK0WNYxxYoaRnV5J8zFNv0ijiBtd3foCjIvmAzmqyDgbJCvQvT7O8kcIr7s6Uv5OOeuaSe-3Ni7iNRMtFD1iA" />

&nbsp;

Build a Date field by filling in the right-side form and selecting &ldquo;Date&rdquo; in the Field Type drop-down menu.

&nbsp;

&nbsp;

<h2>Additional parameters</h2>

<h2>&nbsp;</h2>

If you have additional context about this specific field or how the information is usually displayed in your documents, you can specify it during the field set up.

&nbsp;

&nbsp;

<h3>Feed parameters through UI</h3>

&nbsp;

Through the UI, you may specify a date format between:

<ul>
	<li>US (m/d/y)</li>
	<li>EU (d/m/y)</li>
</ul>

&nbsp;

This format should be the format is which most dates are displayed on your documents.

&nbsp;

Warning: This format is not to be interpreted as the date format you want to have in the API response scheme.

&nbsp;

<h2>&nbsp;</h2>

<h2>Impact for training and making predictions</h2>

&nbsp;

<h3>Candidates</h3>

&nbsp;

By specifying the data type as Date, you will force the engine to only consider words and series of words that can be interpreted as a date as eligible candidates.

&nbsp;

Warning: This means that, for this specific field, we will never be able to extract any information in your document that cannot be interpreted as a date.

&nbsp;

&nbsp;

<h3>Text parsed</h3>

&nbsp;

Anything that can be read as a date including:

<ul>
	<li>Standard formats : &ldquo;12/08/1980&rdquo; or &ldquo;1/1/19&rdquo;</li>
	<li>Alpha characters for months : &ldquo;December, 8th 1980&rdquo;</li>
	<li>Dates missing the day information : &ldquo;January 2019&rdquo;</li>
</ul>

&nbsp;

If the date you are looking for is sometimes displayed in another way, you should use a String field.

&nbsp;

Extract and parse dates (ISO format) from documents.

Dates

<h1>Set up an Email Address field</h1>

&nbsp;

&nbsp;

You want to extract an email address from your documents?

&nbsp;

You can do that with the API Builder by creating an Email Address field in the Data Model configuration step.

&nbsp;

This section will walk you through the implications of setting up a Email Address field.

&nbsp;

<h2>Prerequisites</h2>

&nbsp;

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net">Sign up</a> and confirm your email to login.</li>
</ol>

&nbsp;

&nbsp;

Let&rsquo;s get started!&nbsp;

&nbsp;

After giving a name, description and cover image to your new API, you should land on the following page

&nbsp;

<img src="https://lh4.googleusercontent.com/Wa7NYW16PDbn1zPTQA2EjiASHzi27JHdBq2IOouZKt8K9NFoDGezb3YmS6_NnV4lATuw_nR5yviPLeamTqsFMt8g00EvHOpwCkm_EL1CkR9EFhfZgUv4sNbru3TbZyHB-PPtc21u" />

&nbsp;

&nbsp;

Build an Email Address field by filling in the right-side form and selecting &ldquo;Email address&rdquo; in the Field Type drop-down menu.

&nbsp;

<h2>Additional parameters</h2>

<h2>&nbsp;</h2>

There is no additional parameter to be fed to the engine regarding email addresses.

&nbsp;

<h2>&nbsp;</h2>

<h2>Impact for training and making predictions</h2>

&nbsp;

<h3>Candidates</h3>

&nbsp;

By specifying the data type as Email Address, you will force the engine to only consider words and series of words that can be interpreted as an email address as eligible candidates.

&nbsp;

Warning: This means that, for this specific field, we will never be able to extract any information in your document that cannot be interpreted as an email address.

&nbsp;

Email

Data model

<h1>How it works</h1>

&nbsp;

&nbsp;

This tutorial describes the training cycle of your deep learning algorithm.

&nbsp;

&nbsp;

<h2>Create a dummy API&nbsp;</h2>

&nbsp;

First, we&rsquo;re going to create a dummy API that illustrates the main phases of the training cycle.

&nbsp;

Head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page. <a download="" href="https://mindee-public-website.s3.amazonaws.com/blog/2021/01/14/all_fields.jpg">Here is the image</a> I use for setting up the API, and my setup looks like this:

&nbsp;

&nbsp;

<img src="https://lh3.googleusercontent.com/7mphD2egMHgnL2uKucRn4OvlBHvS1x20r4EUGkkh2e61JRZ6-vO7N3Qprts7FYCJJIDLZ32Ww43CITMePJQH5_yTufsXzXMUN8Fluak3p_a6CX5FU1Br4I3Ob9jcfUkzA9PJARIH" />

&nbsp;

&nbsp;

&nbsp;

Upload <a download="" href="https://mindee-public-website.s3.amazonaws.com/blog/2021/01/14/how_it_works-config">this json</a> into the data model. Once it&rsquo;s done, it should looks like this:

&nbsp;

<img src="https://lh6.googleusercontent.com/Yzm0FkW_zBNQtrztsw8ym0YDjcuRggWKUiXDnbS7utGhdsqC2S5yJyPr-qKmMPZNdtF3dtShaLfVIxV3StvlPHEvqKtZEpcBi4ggJZgq3crxbjNdtleJV_esK7HfZRgooe11MtKh" style="height:428px; width:400px" />

&nbsp;

&nbsp;

In this model, we have set up a field of each type to understand how it works.

&nbsp;

You&rsquo;re all set, we can now click on the &ldquo;start training&rdquo; button.

&nbsp;

&nbsp;

<h2>Training phase</h2>

&nbsp;

Our API was deployed, and you land now on the training section.

&nbsp;

Upload <a href="https://mindee-public-website.s3.amazonaws.com/blog/2021/01/14/all_fields.jpg">this image</a> into the document area:

&nbsp;

&nbsp;

<img src="https://lh3.googleusercontent.com/mddwh3JN8pvKzHWiq24mMWumImhVOxRbLHtGd82SWgOmBT3dnFvWP6PGzXTfagFTfTSfFrCa7fUNzeKSOzrSrqBquibH1GSZEQBtSI82Ln7Sooy4q1pP3d4YK75hBOir-QTCTwAw" />

&nbsp;

&nbsp;

&nbsp;

Click on any field input in the form, you should see the blue boxes on your document changing.

&nbsp;

&nbsp;

<img src="https://lh5.googleusercontent.com/nL_n7ESa3J3IqdwUQSLSIesOldobS3C7csf96S1XqB6X_AZm9I187dkfOE5-p9UZFBi4vMmB4MCknjuNpty3MDBZ_dX2zzkUQdyWKkXKU4ncSBTDVAsLg_F1b451MXhbF9fVh-at" style="width:100%" />

&nbsp;

&nbsp;

The blue boxes displayed on the document for each field are actually the fields candidates. For each field you set up in your data model, our engine will parse the potential candidates so that a user, you, or an automatic script can tell the model what candidate was actually the one you are looking for.

&nbsp;

That&rsquo;s what happens when you click on the validate button.

&nbsp;

For example, let&rsquo;s assume we are looking for the total amount of the invoice.&nbsp;

&nbsp;

When you hit the blue box corresponding to the total amount in the document, the color changes and the amount is filled in the field.

&nbsp;

&nbsp;

<img src="https://lh5.googleusercontent.com/-Jj3HTu733syRhRSWAbD3tp0CCbWFGzNREg3PnEDDTxxSxleeAgBwbrQHloWVL4Z5O3ktaB9ZdD1vOfa1gMv1rfjsKfAZtntWFhh1L3pMz6b1C1YI7Pos_ZIssYk_-q-1Elc5vWj" />

&nbsp;

When you are going to press &ldquo;validate&rdquo;, you are going to tell the deep learning model that this green box was the one corresponding to the total amount. In other words, you are teaching the model which candidate is the total amount, and he&rsquo;s going to learn to extract the right one on new samples.

&nbsp;

At the very beginning, there is no trained model and calling the /predict endpoint will give you only the candidates for each field, no predictions.

&nbsp;

Without any prediction, the /predict endpoint and the training cycle looks like this:

&nbsp;

&nbsp;

<img src="https://lh3.googleusercontent.com/XquXdXE0jeVG_cazVCMiGjpo644aFzxO5VkU6Zhvo09un-rcC-M5-1R3OKtnaKwc1efOzI5RYR1ogU4VDeJBU0QzsMHSiAB0Spv9HD5iKvuQEPSnBmmiXbXXL4opiGLyIjJAvG8f" />

&nbsp;

&nbsp;

The /feedback endpoint is automatically called within the interface, but you can also use the API if you want to do this on your own interface.

 
&nbsp;

<h2>Model training</h2>

&nbsp;

Once there are 20 data sent to the feedback endpoint, the training of your first model will start.

&nbsp;

Our deep learning engine will automatically generate and train a model based on the data you send, and a lot of extra features based on computer visions and natural language processing embeddings.

&nbsp;

In a nutshell, the algorithm will take as inputs the image of each document as well as the whole text. It&rsquo;s then going to be train based on your feedback to extract the right candidates for each field.

&nbsp;

Once the model is trained, it&rsquo;s going to be automatically deployed in your API, and you will see a v1 version of your API appear in the documentation. A new version is released every time a new model is deployed.

&nbsp;

When you call the predict endpoint with a trained model, it&nbsp;will perform an inference before sending back the response:

 
&nbsp;

<img src="https://lh5.googleusercontent.com/qH4l6rtBqsxz9KqmVE6LDixo6sMpeNSNjHOmsTLTlTx-P8jKP5EcfFiAeYms2BPwoHyeaPtO-tUTUVJLxxYjy5ODUUlhVSda4tEY4RmdJf5_Rip_aubUESFjIGqQQ54TMH0Akl6y" />

&nbsp;

&nbsp;

If you want to dive deeper into our product, feel free to reach out to our data scientist team: <a href="mailto:contact@mindee.co" style="text-decoration:none">contact@mindee.co</a> or join the <a href="https://join.slack.com/t/mindeecommunity/shared_invite/zt-kj7i47o6-feQ4TBPFBFTOSv4FDHWHvg" target="_blank">slack channel</a>!

 
&nbsp;

Details about deep learning model generation for document parsing API builder.

How it works

<h1>Generate your client from Swager json</h1>

&nbsp;

&nbsp;

In this tutorial, you&#39;ll learn how to generate your client library using an Open API configuration. We&#39;re going to generate a python client but you can use the same library to generate your client in many languages.

&nbsp;

<h2>Code generation</h2>

&nbsp;

First you will need to download the latest&nbsp;swagger-codegen-cli&nbsp;version from maven central :&nbsp;

<a href="https://repo1.maven.org/maven2/io/swagger/swagger-codegen-cli/" rel="nofollow">https://repo1.maven.org/maven2/io/swagger/swagger-codegen-cli/</a>.

&nbsp;

Make sure to have java installed in your computer by runnning the following command :

&nbsp;

<pre>
<code class="language-bash">foo@bar:~$ java -version

openjdk version "11.0.9.1" 2020-11-04
OpenJDK Runtime Environment (build 11.0.9.1+1-Ubuntu-0ubuntu1.18.04)
OpenJDK 64-Bit Server VM (build 11.0.9.1+1-Ubuntu-0ubuntu1.18.04, mixed mode, sharing)</code></pre>

&nbsp;

Download the&nbsp;swagger.json&nbsp;file from the documentation page of your Mindee project.

&nbsp;

&nbsp;

<img src="https://lh3.googleusercontent.com/GJP52zf3vOLRmXBfYTdKe3VVqVgSyqEYWLqL0YtTCWLuENCT1CgHZPafd7nH2ntfxEzCCYB7Q6RWK_IXcWU0vsuehLbKW_nmSW1jDckpvIml7hwGUMVNVoPzJ1qrh0MAR7Lq69oz" />

&nbsp;

&nbsp;

Replace the &quot;your_swagger.json&quot; with the openai.json file you just downloaded and run the&nbsp;command :

&nbsp;

<pre>
<code>foo@bar:~$ java -jar swagger-codegen-cli-version.jar generate -i your_swagger.json -l python -o your_api_client</code></pre>

&nbsp;

&nbsp;

You can now install your python sdk client :

&nbsp;

<ul>
	<li>Create a virtual env :</li>
</ul>

<pre>
<code class="language-bash">foo@bar:~$ python3 -m venv python_sdk_client_venv
foo@bar:~$ source python_sdk_client_venv/bin/activate</code></pre>

<blockquote>
You can ignore this step if you are already using your own python3 virtual env.
</blockquote>

&nbsp;

<ul>
	<li>Install your python sdk client via setuptools:</li>
</ul>

<pre>
<code>(python_sdk_client_venv) foo@bar:~$ cd your_api_client/
(python_sdk_client_venv) foo@bar:~$ python setup.py install</code></pre>

&nbsp;

&nbsp;

You are all set and ready to use your Python sdk client.

<h3>&nbsp;</h3>

<h3>&nbsp;</h3>

<h2>Usage</h2>

&nbsp;

Create your&nbsp;api key&nbsp;from the API Keys page of your Mindee project:&nbsp;

&nbsp;

<img src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/12/image.png" />

&nbsp;

&nbsp;

&nbsp;

&nbsp;

<h2>Using&nbsp;the predict endpoint</h2>

&nbsp;

&nbsp;

<pre>
<code class="language-python">import swagger_client
from swagger_client.rest import ApiException
# replace XXX by your public api name in camel-case/ 
# for example, if your api is named api_name, replace XXXApi() by ApiNameApi()
api_instance = swagger_client.XXXApi()
authorization = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # your api key
version = "v1" # choose your version
file = "path/to/your/file"
feedback = "true"

try:
 api_response = api_instance.upload_file(
 authorization, version, file, feedback=feedback
 )
except ApiException as e:
 raise e</code></pre>

<pre>

&nbsp;</pre>

&nbsp;

api_response&nbsp;is an object of type&nbsp;swagger_client.models.api_predict_response.ApiPredictResponse.

&nbsp;

<pre>
<code>api_response.input_uuid # if you set feedback to 'true'
api_response.pages
api_response.predictions
</code></pre>

&nbsp;

&nbsp;

<code>api_response.pages.features</code> is a list containing all the candidates the API found for ech field. Each item of the list corresponds to a page:

&nbsp;

<pre>
<code>{
 'pages': {
 'candidates': [
 {
 'feature_1': [
 {
 'content': 'xxxx',
 'key': 'f0311d47',
 'segmentation': {
 'bounding_box': [
 [
 0.2891,
 0.0264
 ],
 [
 0.4479,
 0.0215
 ],
 [
 0.4505,
 0.0635
 ],
 [
 0.2917,
 0.0684
 ]
 ]
 }
 }
 }
 ....
 ],
 'feature_2': ....

 ]
 }
 }
</code></pre>

&nbsp;

&nbsp;

<code>api_response.predictions</code> is a dict: containing the model&#39;s predictions for each feature. Note that if you&#39;re using the v0 version of your API, this object will be set to None as there is no trained model yet.

&nbsp;

<pre>
<code>{
 'feature_1': [
 {'content': 'xxx',
 'key': 'c5d1daf7',
 'page_id': 0,
 'relative_vertices': [[0.0898, 0.0693],
 [0.2708, 0.0635],
 [0.2734, 0.1035],
 [0.0924, 0.1094]]
 }, ....
 ]
 'feature_2': ....
}
</code></pre>

<h4>&nbsp;</h4>

<h4>&nbsp;</h4>

&nbsp;

<h2>Using the feedback endpoint</h2>

&nbsp;

&nbsp;

Sending a feedback requires you to send an image to the /predict endpoint first with the param&nbsp;<code>feedback=true</code>&nbsp;. You&#39;ll then get a &quot;input_uuid&quot; key in your response that you need to feed in your /feedback call.

&nbsp;

<pre>
<code class="language-python">import swagger_client
from swagger_client.rest import ApiException
import json

input_uuid = 'input_uuid_example' # str | Input uuid from /predict endpoint to post the feedback on.
labels = dict() # dict feedback object.
# replace XXX by your public api name in camel-case/ 
# for example, if your api is named api_name, replace XXXApi() by ApiNameApi()
api_instance = swagger_client.XXXApi()
api_feedback_response = api_instance.send_feedback(authorization, input_uuid, json.dumps(labels))</code></pre>

&nbsp;

&nbsp;

labels must have the following format :

&nbsp;

<pre>
<code>{
 'feature_1': {'page_id': 0, 'selected_keys': ['833b42a4', 'a70e7668']},
 'feature_2': ...
}
</code></pre>

&nbsp;

Generate your clients from open API config.

Generate your clients

<h1>Test your document parsing API with Postman</h1>

<h2>&nbsp;</h2>

<h2>Objective</h2>

In this article, we&#39;ll learn how&nbsp;to test a Mindee document parsing API using Postman, a popular developer tool.

&nbsp;

<h2>Prerequisites</h2>

&nbsp;

<ol>
	<li style="list-style-type:decimal">You must have set up a custom document parsing API at <a href="https://beta.mindee.net" style="text-decoration:none" target="_blank">https://beta.mindee.net</a>. If you don&rsquo;t have one ready, set up the Burger Stores demo API described in <a href="https://mindee.com/documentation/beta-api-builder/getting-started" style="text-decoration:none" target="_blank">this tutorial</a>.</li>
	<li style="list-style-type:decimal">For a better developer experience, we recommend that you train your API model with at least 20 documents (as done in the tutorial mentioned above).</li>
	<li style="list-style-type:decimal">The <a href="https://www.postman.com/downloads" style="text-decoration:none" target="_blank">Postman app</a> running on your machine.</li>
</ol>

&nbsp;

<h2>Step 1: Set up Postman</h2>

 
&nbsp;

Assuming you already have a data model ready to go, you can test it right away with Postman (or any other tool that understands the <a href="https://swagger.io/specification/v2/" style="text-decoration:none" target="_blank">OpenAPI v2 specification</a> - fka Swagger v2.0).

&nbsp;

To do so, head over the Documentation page of your Mindee API and select the download openapi.json link, as shown below:

&nbsp;

<img alt="Download OpenAPI from Mindee" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-download-openapi.png" />

&nbsp;

If you followed the <a href="https://mindee.com/documentation/beta-api-builder/getting-started" style="text-decoration:none" target="_blank">Burger Stores and Menus tutorial</a>, you should now have downloaded a file named <code>burger_stores-openapi.json</code>.

&nbsp;

Next, open your Postman desktop client and import the <code>burger_stores-openapi.json</code> file using the Import menu:

&nbsp;

<img src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-postman-import.png" />

&nbsp;

Once you have uploaded your file to Postman, you should see the following confirmation screen:

&nbsp;

<img alt="Postman - model imported" src="https://lh3.googleusercontent.com/1HqIEtCst9we1OvSUGjHwIeGk5yjo3YW-PR037Y1WzvoFmW9B6osGp8JaiBdNWSo5BUNUy9GdMAx-FZqhwUBSclhNI_63BcoyBkvNBQSZTge6_e5_QSbwh874mT3pEyMS-SiY0gz" />

&nbsp;

Press the Import button.&nbsp;Your Burger and Menus Postman collection shows up in Postman, as follows:

&nbsp;

<img alt="Postman - Burger Stores and Menus Collection" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-postman-burger_stores-collection.png" />

Now, let&rsquo;s generate an API key so we can test our API in Postman.

 
&nbsp;

<h2>Step 2: Generate an API key</h2>

 
&nbsp;

Head over the API Keys page of your API Dashboard and press the Create new API key button:

&nbsp;

<img alt="Mindee API Builder - create new API key" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-create-new-api-key.png" />

&nbsp;

Enter the name of the application that will use your API key (such as &ldquo;Postman&rdquo;) and press the Create new API key button. A page similar to the following screenshot&nbsp;appears below:

 
 
 
&nbsp;

<img alt="API Builder API key created" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-created-api-key.png" />

&nbsp;

Next, copy the API key to your clipboard by pressing the copy icon, as shown below:

&nbsp;

<img alt="Mindee API Builder - Copy API key" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-copy-api-key.png" />

&nbsp;

Paste this key right away into a text file, we&rsquo;ll need it in the next step.

&nbsp;

<h2>Step 3: Configure Postman environment variables</h2>

&nbsp;

Back in Postman, create an environment called Burger Stores and a variable called <code>apiToken</code>:

&nbsp;

<img alt="Postman - Burger Stores environment selected" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-postman-create-new-env.png" />

&nbsp;

In the Initial Value column of the <code>apiToken</code> variable, paste the API key you generated in Step 2 and press the Add button.

&nbsp;

Close the Manage Environments window and select the Burger Stores environment:

&nbsp;

<img alt="Postman - create new environment" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-postman-selected-env.png" />

 
Congratulations! Your API is now properly configured in Postman. 
 
In the next article, we&rsquo;ll look at <a href="https://mindeeco.dev.maindev-8jqi1g0x.mindee.net/documentation/api-builder/test-postman/postman-predict-endpoint" style="text-decoration:none">how you can use your API&rsquo;s Predict endpoint</a>.

Configure Postman to test your Mindee API Builder API

<h1>How to use the Predict endpoint with Postman</h1>

&nbsp;

<h2>Objective</h2>

In the previous article, we reviewed how to set up Postman to use a Mindee API Builder API. Each custom API you build and deploy with Mindee has 2 endpoints: <code>/predict </code>and <code>/feedbacks</code>. In this article, we&rsquo;ll deep dive into the <code>/predict&nbsp;</code>endpoint of your API.&nbsp;

&nbsp;

<h2>Prerequisites</h2>

See <a href="https://mindeeco.dev.maindev-8jqi1g0x.mindee.net/documentation/api-builder/test-postman" style="text-decoration:none">previous article</a>

&nbsp;

<h2>Endpoint overview</h2>

&nbsp;

The <code>/predict</code> endpoint takes a document as an input&nbsp;and returns&nbsp;predictions (made by the backing machine learning model)&nbsp;of all fields as defined in your API data model.

&nbsp;

<h2>Calling the Predict endpoint</h2>

&nbsp;

In Postman, select the Predict endpoint and select the Body tab of the request.

&nbsp;

Change the type of the file key from Text to File:

&nbsp;

<img src="https://lh5.googleusercontent.com/jSJ-T8umMXh3_0S3F0sG1ZOQYt-_ea6N5MtrqlveBbBCwqVZnAX7UCHnpGAZwzgGzy4Bd73U2C6LeCy8VvC4hUKVKYBxPxE-Eq8y7udDinf9ibKbdwP_ny9j189AapW1RV5uU37b" style="height:219px; width:602px" />

&nbsp;

In the VALUE column, press the Select Files button and pick an image from the <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/13/burger-stores.zip" style="text-decoration:none">sample training set</a> we used to train the <a href="https://mindee.com/documentation/beta-api-builder/getting-started">Burger Stores and Menus</a> model (try to use an image that wasn&rsquo;t used to train the model):

&nbsp;

<img alt="Postman - File type in Body" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-postman-predict-file.png" />

&nbsp;

Before calling the endpoint, we must perform one last update.

By default, all the Mindee custom APIs are configured to call version 0 of the <code>/predict</code> endpoint. Version 0 is the version that&rsquo;s deployed right after you press the start training&nbsp;button, which means it has no backing machine learning&nbsp;model to make predictions of the most likely candidates for your configured fields. As a result, the <code>/predict</code> endpoint only returns candidates for each field, i.e. the list of potential values for that field, based on its type and optional constraints (such as &ldquo;it&rsquo;s an integer only&rdquo;).

&nbsp;

Since we&rsquo;ve already generated a first model, we can therefore use a more recent version of our Burger Stores and Menus API. Where can you find the most recent version of the API you can use? Head over the Training section of your API and make a note of the active version:

&nbsp;

<img alt="Mindee API Builder - Active version" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-active-version.png" />

&nbsp;

You can also find all the currently available versions in the Params section of your Predict request in Postman:

&nbsp;

<img alt="Mindee API Builder API currents available versions in Postman" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-postman-v0-version.png" />

In the screen above, update the VALUE column of the version key to v1 (instead of v0):

&nbsp;

<img alt="Mindee API Builder v1 version configured in Postman" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-postman-v1-version.png" />

&nbsp;

Press the Send button and observe the response returned by the <code>/burger_stores</code> API (screenshot below with folded <tt>pages.candidates</tt> and <tt>predictions</tt> nodes):

&nbsp;

<img alt="Mindee API Builder - Predict response" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-postman-predict-response.png" />

Note: this is the response structure of the beta release, which will likely evolve in the RTM release.

&nbsp;

<h2>Structure of the /predict endpoint response</h2>

&nbsp;

The <tt>pages.candidates</tt>&nbsp;node is an array where each element of the array represents the page number. For instance, <tt>pages.candidates[0]</tt> represents candidates on the first page, <tt>pages.candidates[1]</tt> on the second page and so on (the beta thus outputs per-page candidates).

&nbsp;

<h3>Candidate node structure</h3>

&nbsp;

The structure of each <code>candidate</code> node is similar to the following one:

&nbsp;

<img alt="Mindee API Builder - Predict candidates in response" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-postman-predict-response-candidates.png" />

In our cast, there is only one such node (since there&rsquo;s only one page in the document we sent to the API), but you can see that all the fields defined in the data model have an entry in each <code>candidate</code> node.

This makes sense since the ML model can identify&nbsp;candidates for a single field in multiple pages. And that&#39;s a desirable behavior.

&nbsp;

<h3>Candidate field node structure</h3>

&nbsp;

Let&rsquo;s drill down further into a field, for instance the name field. Here is an example of a typical structure:

&nbsp;

<img alt="API Builder - Predict response - Name candiidate field" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-postman-predict-response-name-field.png" />

Each field is an array since there can be multiple candidates for that field on the same page.

Each field candidate has a <code>content</code> attribute, the value of which is the OCR&rsquo;ed representation of the parsed text.&nbsp;

The <code>key</code> attribute&nbsp;is the attribute expected by the <code>/feedbacks</code> endpoint when you want to pass a feedback back to Mindee (typically in case when the ML model made an incorrect prediction) so that it will learn from its mistakes and (hopefully) won&rsquo;t make it again.

Last, the segmentation node represents the X,Y coordinates of the blue box (called the bounding_box in the API response). 
&nbsp;

For instance, the bounding box coordinates below:

&nbsp;

<img alt="Mindee Bounding Box" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-postman-predict-response-bounding-box.png" />

&nbsp;

represent the following blue box (read as &ldquo;III&rdquo; when doing a 90-degrees rotation):

&nbsp;

<img alt="Bounding box example on document" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-training-bounding-box.png" />

&nbsp;

<h3>Predictions node structure</h3>

Let&rsquo;s switch to the <code>predictions</code> node, probably the most interesting one for you, since it holds the extracted values the ML model deems the most probable for the document you are trying to parse.

&nbsp;

Here is a typical structure:

&nbsp;

<img alt="Mindee API Builder - Predict Response - Predictions node" src="https://lh5.googleusercontent.com/AWHdOuJ46TaP26WTWbSTU1xEE9ssikNMBAV-71G5E09eqiXutpMBQTHINo8sDRxAAp-JGomk-QHJWlAl112bUiBiPOhrBZN7IhPPpg8LPNXm4TVATHMm56ep3YujwiciiPLJxK16" />

As expected, the API response key of each field (as defined earlier in the <a href="https://mindee.com/documentation/beta-api-builder/getting-started" style="text-decoration:none">Burger Stores and Menus API tutorial</a>) shows up as an array in the <code>predictions</code> node (note that the predictions node is a singleton, not an array, because in the beta we return one set of predictions per document.

&nbsp;

Note that the <code>average_rating</code> field is empty. This means the model wasn&rsquo;t able to find any likely value for that field. Since it obviously exists in the document we submitted, we will see in the next article&nbsp;how we can use the <code>/feedbacks</code> endpoint to submit the correct value for that field.

&nbsp;

<h3>Prediction field node structure</h3>

&nbsp;

A typical structure for a multi-valued field (i.e. a field composed of multiple, consecutive boxes) is the following:

&nbsp;

<img alt="Mindee API Builder - Predict Response - Address field" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-postman-predict-response-address.png" />

&nbsp;

In this case, the first 3 boxes of the address field (as detected by the ML model) are &ldquo;American&rdquo; &ldquo;.&rdquo; and &ldquo;Fast&rdquo; as available&nbsp;in the <code>content</code> keys above.

&nbsp;

Each element in the predicate field array also contains:

<ul>
	<li style="list-style-type:disc">a <code>page_id</code> attribute (referencing the 0-based page number where the element was found),&nbsp;</li>
	<li style="list-style-type:disc">a <code>relative_vertices</code> array of (X,Y) coordinates (similar to the bounding_box element discussed above),</li>
	<li style="list-style-type:disc">a <code>key</code> attribute that uniquely identifies the element (and matches its corresponding candidate element in the <code>candidates</code> array discussed above)</li>
</ul>

&nbsp;

Last but not the least, the <code>input_uuidd</code> attribute contains an id if you call the <code>/predict </code>endpoint with the <code>feedback=true </code>parameter:

&nbsp;

<img alt="Input UUID screenshot" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/29/api-builder-postman-predict_input_uuid.png" />

The value of the <code>input_uuid</code> attribute should be used if you want to send feedback to your API.

That&#39;s it for this long but hopefully useful article. We&rsquo;ll explore in the next tutorial how we can use our knowledge of the <code>/predict</code> endpoint to call the <code>/feedbacks</code> endpoint.

&nbsp;

How to use the Predict Endpoint of an API Builder API with Postman

Predict Endpoint

Test with Postman

<h1>Build your own Document OCR API</h1>

&nbsp;

This article gives you a set of tutorials explaining how to build an accurate&nbsp;OCR API for many different types of documents using Mindee&#39;s&nbsp;engine.

&nbsp;

&nbsp;

If the type of document you want to get an OCR for is not listed here, feel free to reach out to us at&nbsp;<a href="mailto:contact@mindee.com">contact@mindee.com</a>, or using the chat message. &nbsp;

&nbsp;

Here is the list of tutorials to help you get started to build your own OCR:

&nbsp;

<h2>Pay stubs</h2>

<ul>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/us-pay-stubs-ocr">US Pay Stubs</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/french-payslips-ocr">French Payslips</a>
	</li>
</ul>

&nbsp;

<h2>Travel and transportation</h2>

<ul>
	<li><a href="https://mindee.com/documentation/beta-api-builder/document-ocr/boarding-pass-OCR">Boarding Pass</a></li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/train-ticket-ocr">Train Ticket</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/bus-ticket-ocr">Bus Ticket</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/parking-ticket">Parking Ticket</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/traffic-ticket-ocr">Traffic ticket</a>
	</li>
</ul>

&nbsp;

<h2>VISA</h2>

<ul>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/us-visa-ocr">US Visa</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/ds-2019-ocr">DS 2019</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/us-employment-authorization-ocr">US Employment Authorization</a>
	</li>
</ul>

&nbsp;

<h2>Health</h2>

<ul>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/us-medicare-health-insurance-card-ocr">Medicare health insurance card</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/french-health-insurance-cards-ocr">French health insurance card</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/blood-test-results-ocr">Blood Test Results</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/medical-certificate-ocr">Medical Certificate</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/medical-prescription-ocr">Medical prescription</a>
	</li>
</ul>

&nbsp;

<h2>Certificates</h2>

<ul>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/birth-certificates-ocr">Birth certificates</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/marriage-certificate-ocr">Marriage Certificate</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/diploma-ocr">Diploma</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/certificate-origin-ocr">Certificate of Origin</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/certificate-status-ocr">Certificate of Status</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/electricity-bill">Electricity Bill</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/rent-receipt-ocr">Rent Receipt</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/affidavit-residenceocr">Affidavit of Residence</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/bank-statement-ocr">Bank Statement</a>
	</li>
</ul>

&nbsp;

<h2>Shipping</h2>

<ul>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/shippers-letter-instruction-ocr">Shipper&#39;s Letter of Instruction</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/packing-list-ocr">Packing List</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/delivery-note-ocr">Delivery Note</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/bill-lading-ocr">Bill of lading</a>
	</li>
</ul>

&nbsp;

<h2>Tax</h2>

<ul>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/1040-forms-ocr">1040 forms</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/w9-ocr">W9</a>
	</li>
</ul>

&nbsp;

<h2>Others</h2>

<ul>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/energy-performance-diagnostic">Energy performance diagnostic</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/nutrition-facts-label-ocr">Nutrition facts label</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/bank-checks-ocr">Bank checks</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/ss4-ocr">SS4</a>
	</li>
	<li>
	<a href="https://mindee.com/documentation/beta-api-builder/document-ocr/commercial-general-liability-section-ocr">Commercial General Liability Section</a>
	</li>
</ul>

&nbsp;

&nbsp;

Example list of supported document types you can build using Mindee OCR API builder.

<h1>US Pay Stubs OCR</h1>

&nbsp;

&nbsp;

This article lays out the process recommended to build an OCR API that extracts data from US pay stubs using Mindee&#39;s deep learning engine.

&nbsp;

<h2>&nbsp;</h2>

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 US pay stubs (images or PDFs) to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your Pay Stub use case</h2>

&nbsp;

You might need to automatically extract data from pay stubs to improve your user experience in payroll or loan eligibility workflows. This article will guide you over the few steps required to deploy your Pay Stubs data extraction API.

&nbsp;

&nbsp;

First, we&rsquo;re going to define the fields we want to extract from your pay stubs.

&nbsp;

<img alt="Pay stub OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-4.png" />

&nbsp;

Here is the list of fields we are going to extract using our OCR API:

&nbsp;

<ul>
	<li>Employer: The full name of the employer issuing the pay stub</li>
	<li>Net pay: Total net paid to the employee</li>
	<li>Pay date: Date of wage payment</li>
	<li>Period beginning: Pay stub start date</li>
	<li>Period ending: Pay stub end date</li>
	<li>Gross pay: Total gross pay before taxes and deductions</li>
	<li>Total tax: Total tax deducted</li>
</ul>

&nbsp;

&nbsp;

You can add as many relevant fields as you need to better fit&nbsp;your requirements.

&nbsp;

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined what fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page.&nbsp;<a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/pay_stub_example.jpg" target="_blank">Here is the image</a>&nbsp;you can use to set up the API. For instance,&nbsp;my setup is as follows:

&nbsp;

&nbsp;

<img alt="Setup your Pay stub OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-5.png" />

&nbsp;

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract.

 
&nbsp;

To move forward, you can <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/pay_stubs-config" download="us-paystubs-config.json" target="_blank">download this json config</a> to set up your data model, or you can do it manually.

&nbsp;

&nbsp;

Employer: type String&nbsp;

&nbsp;

<img alt="Employer field for Pay Stub OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-6.png" />

&nbsp;

&nbsp;

Net pay: type Amount

&nbsp;

<img alt="Net paid field for Pay Stub OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-7.png" />

&nbsp;

Pay date: type Date

&nbsp;

<img alt="Pay date field for Pay Stub OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-8.png" />

&nbsp;

&nbsp;

Period beginning: type Date&nbsp;

&nbsp;

<img alt="Pay stub beginning of period field for Pay stub OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-9.png" />

&nbsp;

&nbsp;

Period ending: type Date

<img alt="Pay stub end of period field for Pay stub OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-10.png" />

&nbsp;

&nbsp;

Gross pay: type Amount

&nbsp;

<img alt="Total gross paid for Pay Stub OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-11.png" />

&nbsp;

&nbsp;

Total tax: Total tax deducted

&nbsp;

<img alt="Total tax field for Pay stub OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-12.png" />

&nbsp;

&nbsp;

&nbsp;

<h2>Train your Pay Stub&nbsp;OCR</h2>

&nbsp;

<img src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/19/image.png" />

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

Now is the time to train your US Pay Stub&nbsp;deep learning model in the Training section of your API.&nbsp;

&nbsp;

To get more information about the training phase, please refer to the <a href="https://mindee.com/documentation/beta-api-builder/getting-started">Getting Started tutorial</a>.

If you have any questions regarding your use case, feel free to reach out on the <a href="http://bit.ly/36gSvO9" target="_blank">Mindee Community on Slack</a>!

How to build an accurate Pay Stubs OCR using Mindee's deep learning engine.

US Pay Stubs

<h1>French Payslip OCR 
Bulletins de salaire</h1>

&nbsp;

&nbsp;

This article explains how to build an OCR API that automatically extracts&nbsp;data from French payslips (bulletins de salaire).&nbsp;

&nbsp;

<h2>&nbsp;</h2>

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 French payslips images or PDFs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

&nbsp;

<h2>Define your French payslips use case</h2>

&nbsp;

First, we need to specify the fields we want to extract from our payslips.

&nbsp;

&nbsp;

<img alt="French Payslip OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-44.png" />

&nbsp;

&nbsp;

For our example,&nbsp;we are going to extract the following list of fields from our French payslips:

&nbsp;

<ul>
	<li>Employee full name: First and last names of the employee</li>
	<li>Employee SSN: Employee social security number</li>
	<li>Employer SIRET: Employer SIRET number&nbsp;</li>
	<li>Payslip period: &nbsp;Payslip month and year&nbsp;</li>
	<li>Net paid: Total net paid&nbsp;</li>
	<li>Gross salary:&nbsp;Total gross salary before taxes</li>
</ul>

&nbsp;

Feel free to add any data you&#39;d like the OCR to extract.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined the fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You now land now on the setup page.&nbsp;<a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/fiche-de-paie-2020.jpg" target="_blank">Here is the image</a>&nbsp;you can use to set up the API, and my setup looks like this:

&nbsp;

&nbsp;

<img alt="setup french pyslip ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-45.png" />

&nbsp;

&nbsp;

We&#39;re ready! Press the &ldquo;next step&rdquo; button. We are going to build our data model in the next section.

 
&nbsp;

At this point, you can manually add each field as described below&nbsp;or you can&nbsp;<a download="french_payslip-config.json" href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/french_payslip-config">download this json config</a>&nbsp;and upload it in the left section&nbsp;of the screen.

&nbsp;

&nbsp;

Employee full name: type String with no numeric characters

&nbsp;

<img alt="employee full name payslip ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-46.png" />

&nbsp;

&nbsp;

Employee SSN: type String. Note that&nbsp;we haven&#39;t checked the &quot;It never contains alpha characters&quot; as social security numbers can contain &#39;a&#39; or &#39;b&#39; for Corsican.

&nbsp;

<img alt="employee ssn payslip ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-47.png" />

&nbsp;

&nbsp;

Employer company SIRET: type String that never contains alpha characters.

&nbsp;

<img alt="company siret payslip ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-48.png" />

&nbsp;

&nbsp;

Payslip period: type Date&nbsp;

&nbsp;

<img alt="payslip date ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-49.png" />

&nbsp;

&nbsp;

Net paid: type Amount

&nbsp;

<img alt="Net paid payslip ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-50.png" />

&nbsp;

&nbsp;

Gross salary: type Amount

&nbsp;

<img alt="Gross salary payslip OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-51.png" />

&nbsp;

&nbsp;

<h2>Train your Payslip&nbsp;OCR</h2>

&nbsp;

<img src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-43.png" />

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

Now is the time to train your custom Payslip deep learning model. To get more information about the training phase, please refer to the <a href="https://mindee.com/documentation/beta-api-builder/getting-started">Getting Started tutorial</a>. And if you have any questions regarding your use case, feel free to reach out to us on our&nbsp;<a href="http://bit.ly/36gSvO9" target="_blank">Mindee Community on Slack</a>!

How to build an accurate French payslips (bulletins de salaries) OCR using Mindee's deep learning engine.

French Payslips

<h1>Boarding Pass&nbsp;OCR</h1>

&nbsp;

This article walks you through the building process of&nbsp;an OCR API that extracts data from Flight Boarding Passes using our deep learning engine. It will work for any airline company or boarding pass template.&nbsp;

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 Boarding Pass&nbsp;images or PDFs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your Boarding Pass use case</h2>

&nbsp;

First, we&rsquo;re going to define the fields we&#39;d like to extract from a Boarding Pass.&nbsp;

&nbsp;

&nbsp;

<img alt="Boarding Pass OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-143327.png" />

<ul>
	<li>Passenger name: The full name of the travelling&nbsp;passenger (John Doe)</li>
</ul>

&nbsp;

<ul>
	<li>Flight number: The flight number for the boarding pass, including the airline&#39;s identification letters (AC 2505)</li>
</ul>

&nbsp;

<ul>
	<li>Date: The date of departure in US format (10/12/2017)</li>
</ul>

&nbsp;

<ul>
	<li>Departure airport: The airport of departure including the airport identification letters (New Delhi DEL)</li>
</ul>

&nbsp;

<ul>
	<li>Destination airport: The airport of destination&nbsp;including the airport identification letters (Los Angeles LAX)</li>
</ul>

&nbsp;

<ul>
	<li>Boarding Time: The boarding time (07:45)</li>
</ul>

&nbsp;

&nbsp;

That&rsquo;s it for our use case. Feel free to add any other relevant data to fit your requirements.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined the list of fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You now land on the API&nbsp;setup page. <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/https___prodstatic9netau___media_2018_04_19_10_39_boardingpass.jpg">Here is the image</a>&nbsp;you can use to set up the API - for example, my setup looks as follows:

&nbsp;

&nbsp;

<img alt="Set up your Boarding Pass OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-141914.png" />

&nbsp;

Once you&rsquo;re ready, press the &ldquo;next step&rdquo; button and let&#39;s&nbsp;specify the data types for each of the fields we want our API to extract.

 
&nbsp;

To move forward with your model creation, you can <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/boarding_pass-config.json">download this json config</a> to set up your data model or do it manually.

 
&nbsp;

Passenger name: type String that never contains numeric characters.

&nbsp;

<img alt="Passenger Name field for Boarding Pass OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-142554.png" />

&nbsp;

&nbsp;

&nbsp;

Flight number: type String&nbsp;without specifications.&nbsp;

&nbsp;

<img alt="Flight Number field for Boarding Pass OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-142810.png" />

&nbsp;

Date: type Date with US format order.

&nbsp;

<img alt="Date field for Boarding Pass OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-142936.png" />

&nbsp;

Departure airport:&nbsp;type String that never contains numeric characters.

&nbsp;

<img alt="Departure airport field for Boarding Pass OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-143044.png" />

&nbsp;

Destination airport: type String that never contains numeric characters.

&nbsp;

<img alt="Destination airport field for Boarding Pass OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-143106.png" />

&nbsp;

&nbsp;

Date: type String that never contains alpha characters

&nbsp;

<img alt="Boarding time field for boarding pass OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-143151.png" />

&nbsp;

&nbsp;

Once you&rsquo;re done setting up your data model, press the Data model ready? Deploy your API&nbsp;button at the bottom of the screen.

&nbsp;

&nbsp;

<h2>Train your Boarding Pass&nbsp;OCR model</h2>

&nbsp;

&nbsp;

<img alt="Deep learning Boarding Pass OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-143314.png" />

&nbsp;

&nbsp;

&nbsp;

Now is the time to train your Boarding Pass deep learning model in the Training section of your API.&nbsp;

&nbsp;

&nbsp;

In a few hours (minutes if you&#39;re expeditious), you&rsquo;ll get your first model trained and will be able to use your custom OCR API for parsing Boarding Passes in your application.

&nbsp;

To get more information about the training phase, please refer to the <a href="https://mindee.com/documentation/beta-api-builder/getting-started">Getting Started tutorial</a>.

If you have any questions regarding your use case, feel free to reach out on the <a href="http://bit.ly/36gSvO9" target="_blank">Mindee Community on Slack</a>!

&nbsp;

&nbsp;

How to build an accurate Boarding Pass OCR using Mindee's deep learning engine.

Boarding Pass

<h1>Train Ticket OCR</h1>

&nbsp;

This article shows&nbsp;you how to build&nbsp;an OCR API that extracts data from train tickets (or eTickets) using our deep learning engine. It will work for any train ticket format&nbsp;or template, paper or digital. Improve your workflow by extracting data directly from your train ticket documents.&nbsp;

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 train ticket images or PDFs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your Train Ticket&nbsp;use case</h2>

&nbsp;

First, we&rsquo;re going to define the fields we&#39;d like&nbsp;to extract from our&nbsp;train tickets.&nbsp;

&nbsp;

&nbsp;

<img alt="Train ticket OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/aaa.png" />

&nbsp;

&nbsp;

<ul>
	<li>Passenger name: The full name of the passenger traveling (Jane Martin)</li>
</ul>

&nbsp;

<ul>
	<li>Reservation number: The reservation number of your train ticket&nbsp;&nbsp;(925D90)</li>
</ul>

&nbsp;

<ul>
	<li>Departure Date: The date of departure&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Departure station code: The 3 letters identification code for your departure station &nbsp;(WAS)</li>
</ul>

&nbsp;

<ul>
	<li>Arrival station code: The 3 letters identification code for your arrival station &nbsp;(NYP)</li>
</ul>

&nbsp;

<ul>
	<li>Departure Time: The train departure time from the departure station (01:50 PM)</li>
</ul>

&nbsp;

&nbsp;

That&rsquo;s it for our use case. Feel free to add any other relevant data to fit your requirements.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined all of the fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You now land on the setup page. <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/train-ticket.jpeg">Here is the image</a> you can use for setting up the API, and my setup looks like this:

&nbsp;

&nbsp;

<img alt="Set up your Train Ticket OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/bbb.png" />

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract.

 
&nbsp;

To go further, you can <a class="force-download" download="train_ticket-config.json" href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/train_ticket-config.json?forcedownload=1">download this json config</a>&nbsp;to automatically set up your data model by pressing the Upload a json config button or by building it manually.

 
&nbsp;

Passenger name: String type that never contains numeric characters.

&nbsp;

<img alt="Passenger name field for train ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/ccc.png" />

&nbsp;

&nbsp;

&nbsp;

Reservation number: String type without specifications.&nbsp;

&nbsp;

<img alt="Reservation number field for train ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/ddd.png" />

&nbsp;

Departure Date: Date type as String without specifications.

&nbsp;

<img alt="Departure date field for train ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/eee.png" />

&nbsp;

Departure Station Code:&nbsp;String type&nbsp;that never contains numeric characters.

&nbsp;

<img alt="Departure station code field for train ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/fff.png" />

&nbsp;

Arrival Station Code: type String that never contains numeric characters.

&nbsp;

<img alt="Arrival station code field for train ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/ggg.png" />

&nbsp;

&nbsp;

Departure Time: type String without specifications.

&nbsp;

<img alt="departure time field for train ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/hhh.png" />

&nbsp;

&nbsp;

Once you&rsquo;re done setting up your data model, press the Data model ready? Deploy your API&nbsp;button at the top of the screen.

&nbsp;

&nbsp;

<h2>Train your Boarding Pass&nbsp;OCR</h2>

&nbsp;

&nbsp;

<img alt="Deep learning train ticket OCR training" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/iii.png" />

&nbsp;

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

Now is the time to train your train ticket deep learning model in the Training section of your API.&nbsp;

&nbsp;

In a few hours (minutes if you&#39;re swift), you&rsquo;ll get your first model trained and will be able to use your custom OCR API for parsing train tickets in your application.

&nbsp;

To get more information about the training phase, please refer to the <a href="https://mindee.com/documentation/beta-api-builder/getting-started">Getting Started tutorial</a>.

If you have any questions regarding your use case, feel free to reach out on the <a href="http://bit.ly/36gSvO9" target="_blank">Mindee Community on Slack</a>!

&nbsp;

How to build an accurate Train Ticket OCR using Mindee's deep learning engine.

Train Ticket

<h1>Bus Ticket OCR</h1>

&nbsp;

This article walks you through the building process of&nbsp;an OCR API that extracts data from Bus Ticket&nbsp;using our deep learning engine. It will work for any bus company.

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 Bus Ticket images or pdfs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your Bus Ticket use case</h2>

&nbsp;

First, we&rsquo;re going to define what fields we want to extract from your Bus Ticket.&nbsp;

&nbsp;

&nbsp;

<img alt="Bus Ticket OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-172506.png" />

&nbsp;

&nbsp;

<ul>
	<li>Bus Company: The name of the company that sold the ticket&nbsp;(megabus.com)</li>
</ul>

&nbsp;

<ul>
	<li>Reservation Number: The reservation number that will be asked when you board the bus (67-2722-033018-M10R-1415-LAS-ANA)</li>
</ul>

&nbsp;

<ul>
	<li>Date: The date of departure (March 30, 2018)</li>
</ul>

&nbsp;

<ul>
	<li>Departure city: The city of departure for your travel by bus&nbsp;(Las Vegas)</li>
</ul>

&nbsp;

<ul>
	<li>Arrival city: The city of destination (Anaheim)</li>
</ul>

&nbsp;

<ul>
	<li>Departure Time: The bus departure&nbsp;time from the departure city (2:15 PM)</li>
</ul>

&nbsp;

<ul>
	<li>Price: The total amount paid for the bus ticket ($41,75)</li>
</ul>

&nbsp;

&nbsp;

That&rsquo;s it for our use case, but you are&nbsp;free to add any other relevant data to fit your requirements.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined the list of fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page. <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/megabus.jpeg">Here is the image</a> you can use for setting up the API, and my setup looks like this:

&nbsp;

&nbsp;

<img alt="Set up your bus ticket OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/2.png" />

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract from your bus ticket.

 
&nbsp;

To go further, you can <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/bus_ticket-config-1.json">download this json config</a> to set up your data model or do it manually.

 
&nbsp;

Bus company: type String that never contains numeric characters.

&nbsp;

<img alt="Bus company field for bus ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/3.png" />

&nbsp;

&nbsp;

&nbsp;

Reservation number: type String&nbsp;without specifications.&nbsp;

&nbsp;

<img alt="Reservation Number field for bus ticket OCR " src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/4.png" />

&nbsp;

Date: type String without specifications.

&nbsp;

<img alt="Date field for bus ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/5.png" />

&nbsp;

Departure city:&nbsp;type String that never contains numeric characters.

&nbsp;

<img alt="Departure city field for bus ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/6.png" />

&nbsp;

Arrival city: type String that never contains numeric characters.

&nbsp;

<img alt="Arrival city field for bus ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/7.png" />

&nbsp;

&nbsp;

Departure Time: type String without specifications.&nbsp;

&nbsp;

<img alt="Departure time field for bus ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/8.png" />

&nbsp;

&nbsp;

Price: type Number without specifications.&nbsp;

&nbsp;

<img alt="Price field for bus ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/9.png" />

&nbsp;

&nbsp;

Once you&rsquo;re done setting up your bus ticket data model, press the Data model ready? start training button at the top of the screen.

&nbsp;

&nbsp;

<h2>Train your Bus Ticket OCR</h2>

&nbsp;

&nbsp;

<img alt="Deep learning Bus ticket OCR training" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-172451.png" />

&nbsp;

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

Now is the time to train your Bus ticket deep learning model in the Training section of our API.&nbsp;

&nbsp;

In a few hours (minutes if you&#39;re fast), you&rsquo;ll get your first model trained and will be able to use your custom OCR API for parsing Bus Ticket in your application.

To get more information about the training phase, please refer to the getting started tutorial. If you have any questions regarding your use case, feel free to reach out!

&nbsp;

How to build an accurate Bus Ticket OCR using Mindee's deep learning engine.

Bus Ticket

<h1>Parking Ticket OCR</h1>

&nbsp;

This article walks you through the building process of&nbsp;an OCR API that extracts data from Parking Ticket using our deep learning engine.

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 Parking Ticket images or pdfs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your Parking Ticket use case</h2>

&nbsp;

First, we&rsquo;re going to define what fields we want to extract from your Parking Ticket.&nbsp;

&nbsp;

&nbsp;

<img alt=" Parking Ticket OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/wwww.png" />

&nbsp;

<ul>
	<li>Citation Number: The citation identification number&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Date: The date of the citation</li>
</ul>

&nbsp;

<ul>
	<li>Time: The time it was when the citation was issued</li>
</ul>

&nbsp;

<ul>
	<li>Licence Number: The licence number of the vehicle mentioned in the citation&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>State: The state in which the citation was delivered</li>
</ul>

&nbsp;

<ul>
	<li>Amount Due: The total amount due by the vehicle&#39;s owner</li>
</ul>

&nbsp;

&nbsp;

That&rsquo;s it for our use case. Feel free to add any other relevant data to fit your requirements.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined the list of fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page. <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/parking-ticket.png">Here is the image </a>you can use for setting up the API, and my setup looks like this:

&nbsp;

&nbsp;

<img alt="Set up your Parking Ticket OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/xxxx.png" />

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract.

 
&nbsp;

To go further, you can <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/parking_ticket-config.json">download this json config</a> to set up your data model or do it manually.

 
&nbsp;

Citation Number: type Number without specifications.&nbsp;

&nbsp;

<img alt="Citation Number field for Parking Ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/yyyy.png" />

&nbsp;

&nbsp;

&nbsp;

Date: type Date with US format.

&nbsp;

<img alt="Date field for Parking Ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/zzzz.png" />

&nbsp;

Time issued: type String without specifications.&nbsp;

&nbsp;

<img alt="Time field for Parking Ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/zzzza.png" />

&nbsp;

Licence Number:&nbsp;type String without specifications.&nbsp;

&nbsp;

<img alt="Licence Number for Parking Ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/zzzzb.png" />

&nbsp;

State: type String that never contains numeric characters.&nbsp;

&nbsp;

<img alt="State field for Parking Ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/zzzzc.png" />

&nbsp;

&nbsp;

Amount Due: type Number without specifications.&nbsp;

&nbsp;

<img alt="Amount Due for Parking Ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/zzzzd.png" />

&nbsp;

&nbsp;

Once you&rsquo;re done setting up your data model, press the Data model ready? start training button at the top of the screen.

&nbsp;

&nbsp;

&nbsp;

<h2>Train your Parking Ticket OCR</h2>

&nbsp;

&nbsp;

<img alt="Deep learning Parking Ticket OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/zzzze.png" />

&nbsp;

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

Now is the time to train your Parking Ticket deep learning model in the Training section of our API.&nbsp;

&nbsp;

In a few hours (minutes if you&#39;re fast), you&rsquo;ll get your first model trained and will be able to use your custom OCR API for parsing Parking Ticket in your application.

&nbsp;

To get more information about the training phase, please refer to the getting started tutorial. If you have any question regarding your use case, feel free to reach out!

&nbsp;

&nbsp;

How to build an accurate Parking Ticket OCR using Mindee's deep learning engine.

Parking Ticket

<h1>Traffic ticket OCR</h1>

&nbsp;

This article describes how to build an OCR API&nbsp;that extracts data from Traffic ticket using our deep learning engine. If you want to automate your traffic ticket workflow, this article is for you.&nbsp;

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 Traffic ticket images or pdfs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your Traffic ticket use case</h2>

&nbsp;

First, we&rsquo;re going to define what fields we want to extract from your Traffic ticket.&nbsp;

&nbsp;

&nbsp;

<img alt=" Traffic Ticket OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/capture-decran-2021-01-21-a-163834.png" />

&nbsp;

<ul>
	<li>Date: The date of the offence</li>
</ul>

&nbsp;

<ul>
	<li>Time: The time of the offence</li>
</ul>

&nbsp;

<ul>
	<li>Description: The description of the violation</li>
</ul>

&nbsp;

<ul>
	<li>Place of occurrence: The place where the violation occurred</li>
</ul>

&nbsp;

<ul>
	<li>Badge: The badge number</li>
</ul>

&nbsp;

That&rsquo;s it for our use case. Feel free to add any other relevant data to fit your requirements. For example, you could add the county, the arrest type, or anything else you want really.&nbsp;

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined the list of fields you want to extract from your traffic ticket document, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page. <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/proces-verbal.png">Here is the image </a>you can use for setting up the API, and my setup looks like this:

&nbsp;

&nbsp;

<img alt="Set up your Traffic Ticket OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/grege.png" />

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract.

 
&nbsp;

To go further, you can <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/traffic_ticket-config.json">download this json config</a> to set up your data model or do it manually.

 
&nbsp;

Date of Offence: type Date with US format

&nbsp;

<img alt="Date field for Traffic Ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/bgbgb.png" />

&nbsp;

Time: type String without specifications.&nbsp;

&nbsp;

<img alt="Time field for Traffic Ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/fdfgfdgf.png" />

&nbsp;

Description of Violation: type String without specifications.&nbsp;

&nbsp;

<img alt="Description field for Traffic Ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/capture-decran-2021-01-21-a-163735.png" />

&nbsp;

Place of Occurrence:&nbsp;type String without specifications.&nbsp;

&nbsp;

<img alt="Place of Occurence for Traffic Ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/capture-decran-2021-01-21-a-163747.png" />

&nbsp;

Badge Number: type Number without specifications.&nbsp;

&nbsp;

<img alt="Badge number field for Traffic Ticket OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/capture-decran-2021-01-21-a-163759.png" />

&nbsp;

&nbsp;

Once you&rsquo;re done setting up your data model, press the Data model ready? start training button at the top of the screen.

&nbsp;

&nbsp;

<h2>Train your Traffic ticket OCR</h2>

&nbsp;

&nbsp;

<img alt="Deep learning Traffic Ticket OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/capture-decran-2021-01-21-a-163831.png" />

&nbsp;

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

Now is the time to train your Traffic ticket deep learning model in the Training section of our API.&nbsp;

&nbsp;

&nbsp;

In a few hours (minutes if you&#39;re fast), you&rsquo;ll get your first model trained and will be able to use your custom OCR API for parsing Traffic ticket in your application.

&nbsp;

To get more information about the training phase, please refer to the getting started tutorial. If you have any question regarding your use case, feel free to reach out!

&nbsp;

How to build an accurate Traffic ticket OCR using Mindee's deep learning engine.

Traffic ticket

<h1>US Visa OCR</h1>

&nbsp;

This article walks you through the building process of&nbsp;an OCR API that extracts data from US Visas using our deep learning engine. It will work for any visa issued by the United States of America.&nbsp;&nbsp;

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 US Visa images or pdfs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your US Visa use case</h2>

&nbsp;

First, we&rsquo;re going to define what fields we want to extract from your US Visa.&nbsp;

&nbsp;

&nbsp;

<img alt="US Visa OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-170254.png" />

&nbsp;

&nbsp;

<ul>
	<li>First name: The first name of the visa holder (Happyperson)</li>
</ul>

&nbsp;

<ul>
	<li>Last name: The last&nbsp;name of the visa holder (Traveler)</li>
</ul>

&nbsp;

<ul>
	<li>Passport number: The passport number of the visa holder (AB1234567)</li>
</ul>

&nbsp;

<ul>
	<li>Control Number:&nbsp;&nbsp;The visa control number&nbsp;(20191234567890)</li>
</ul>

&nbsp;

<ul>
	<li>Visa type: The visa type appearing on the US Visa&nbsp;(B1/B2)</li>
</ul>

&nbsp;

&nbsp;

That&rsquo;s it for our use case. Feel free to add any other relevant data to fit your requirements.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined the list of fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page. <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/via.png">Here is the image</a> you can use for setting up the API, and my setup looks like this:

&nbsp;

&nbsp;

<img alt="Set up your US Visa OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-165700.png" />

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract.

 
&nbsp;

To go further, you can <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/us_visa-config.json">download this json config</a> to set up your data model or do it manually.

 
&nbsp;

First name: type String that never contains numeric characters.

&nbsp;

<img alt="First name field for US Visa OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-165725.png" />

&nbsp;

&nbsp;

&nbsp;

Last Name: type String&nbsp;without specifications.&nbsp;

&nbsp;

<img alt="Last name field for US Visa OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-165758.png" />

&nbsp;

Passport Number: type String without specifications.

&nbsp;

<img alt="Passport number field for US Visa OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-165814.png" />

&nbsp;

Control Number:&nbsp;type Number without specifications.&nbsp;

&nbsp;

<img alt="Control number field for US Visa OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-165831.png" />

&nbsp;

Visa Type: type String without specifications.

&nbsp;

<img alt="Visa type field for US Visa OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-165854.png" />

&nbsp;

&nbsp;

Once you&rsquo;re done setting up your data model, press the Data model ready? start training button at the top of the screen.

&nbsp;

&nbsp;

<h2>Train your US Visa OCR</h2>

&nbsp;

&nbsp;

<img alt="Deep learning US Visa OCR training" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-171901.png" />

&nbsp;

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

Now is the time to train your US Visa deep learning model in the Training section of our API.&nbsp;

&nbsp;

&nbsp;

In a few hours (minutes if you&#39;re fast), you&rsquo;ll get your first model trained and will be able to use your custom OCR API for parsing US Visa in your application.

&nbsp;

To get more information about the training phase, please refer to the getting started tutorial. If you have any questions regarding your use case, feel free to reach out!

&nbsp;

How to build an accurate US VISA OCR using Mindee's deep learning engine.

US Visa

<h1>DS 2019 form OCR</h1>

&nbsp;

This article walks you through the building process of&nbsp;an OCR API that extracts data from the DS 2019 form using our deep learning engine. It will work for any airline company or boarding pass template.&nbsp;

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 DS 2019 form images or pdfs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your DS 2019 use case</h2>

&nbsp;

First, we&rsquo;re going to define what fields we want to extract from your DS 2019 form.&nbsp;

&nbsp;

&nbsp;

<img alt="DS 2019 OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-180213.png" />

<ul>
	<li>First Name: The first name of the DS 2019 holder</li>
</ul>

&nbsp;

<ul>
	<li>Family Name: The last name of the DS 2019 holder</li>
</ul>

&nbsp;

<ul>
	<li>Gender: The DS 2019 holder gender&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Visa type: The visa type linked to the DS 2019 form</li>
</ul>

&nbsp;

<ul>
	<li>End of cover period: The ending date&nbsp;of the DS 2019 cover period</li>
</ul>

&nbsp;

That&rsquo;s it for our use case. Feel free to add any other relevant data to fit your requirements.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined the list of fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page. <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/ds-2019.png">Here is the image</a> you can use for setting up the API, and my setup looks like this:

&nbsp;

&nbsp;

<img alt="Set up your DS 2019 form OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-175851.png" />

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract.

 
&nbsp;

To go further, you can <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/ds_2019-config.json">download this json config </a>to set up your data model or do it manually.

 
&nbsp;

First name: type String that never contains numeric characters.

&nbsp;

<img alt="First name field for DS 2019 OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-175939.png" />

&nbsp;

&nbsp;

&nbsp;

Family name: type String that never contains numeric characters.

&nbsp;

<img alt="Family name field for DS 2019 OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-175923.png" />

&nbsp;

Gender: type String that never contains numeric characters.&nbsp;&nbsp;

&nbsp;

<img alt="Gender field for DS 2019 OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-175955.png" />

&nbsp;

Visa Type:&nbsp;type String without specifications.

&nbsp;

<img alt="Visa type field for DS 2019 OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-181238.png" />

&nbsp;

End of cover period: type Date in US format.

&nbsp;

<img alt="End of cover period field DS 2019 OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-180108.png" />

&nbsp;

&nbsp;

&nbsp;

Once you&rsquo;re done setting up your data model, press the Data model ready? start training button at the top of the screen.

&nbsp;

&nbsp;

<h2>Train your DS 2019 form OCR</h2>

&nbsp;

&nbsp;

<img alt="Deep learning DS 2019 OCR training" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-180201.png" />

&nbsp;

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

Now is the time to train your DS 2019&nbsp;deep learning model in the Training section of our API.&nbsp;

&nbsp;

&nbsp;

In a few hours (minutes if you&#39;re fast), you&rsquo;ll get your first model trained and will be able to use your custom OCR API for parsing DS 2019 form in your application.

&nbsp;

To get more information about the training phase, please refer to the getting started tutorial. If you have any questions regarding your use case, feel free to reach out!

&nbsp;

How to build an accurate DS 2019 OCR using Mindee's deep learning engine.

DS 2019

<h1>US Employment Authorization OCR</h1>

&nbsp;

This article walks you through the building process of&nbsp;an OCR API that extracts data from US Employment Authorization&nbsp;using our deep learning engine.&nbsp;

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 US Employment Authorization&nbsp;images or pdfs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your US Employment Authorization use case</h2>

&nbsp;

First, we&rsquo;re going to define what fields we want to extract from your US Employment Authorization.&nbsp;

&nbsp;

&nbsp;

<img alt="US Employment Authorization OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-173525.png" />

&nbsp;

&nbsp;

<ul>
	<li>Surname: The last name of the work permit holder&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Given Name: The first name of the work permit holder&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Category: The US Employment Authorization category&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Valid from: The starting date of the validity of your US employment authorization card&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Card expires: The expiration date of your US employment authorization card</li>
</ul>

&nbsp;

&nbsp;

That&rsquo;s it for our use case. Feel free to add any other relevant data to fit your requirements.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined the list of fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page. <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/work-permit.png">Here is the image </a>you can use for setting up the API, and my setup looks like this:

&nbsp;

&nbsp;

<img alt="Set up your US Employment Authorization OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-173138.png" />

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract.

 
&nbsp;

To go further, you can <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/us_employment_authorization-config.json">download this json config </a>to set up your data model or do it manually.

 
&nbsp;

Surname: type String that never contains numeric characters.

&nbsp;

<img alt="Surname field for US Employment Authorization OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-173300.png" />

&nbsp;

&nbsp;

&nbsp;

Given Name: type String&nbsp;that never contains numeric characters.&nbsp;&nbsp;

&nbsp;

<img alt="Given name field for US Employment Authorization OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-173311.png" />

&nbsp;

Category: type String without specifications.&nbsp;

&nbsp;

<img alt="Category field for US Employment Authorization OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-173331.png" />

&nbsp;

Valid from:&nbsp;type Date in US format.&nbsp;

&nbsp;

<img alt="Valid from field for US Employment Authorization OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-173404.png" />

&nbsp;

Card expires&nbsp;type Date in US format.

&nbsp;

<img alt="Card expires field for US Employment Authorization OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-173419.png" />

&nbsp;

&nbsp;

Once you&rsquo;re done setting up your data model, press the Data model ready? start training button at the top of the screen.

&nbsp;

&nbsp;

<h2>Train your US Employment Authorization&nbsp;OCR</h2>

&nbsp;

&nbsp;

<img alt="Deep learning US Employment Authorization OCR training" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/20/capture-decran-2021-01-20-a-173632.png" />

&nbsp;

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

Now is the time to train your US Employment Authorization&nbsp;deep learning model in the Training section of our API.&nbsp;

&nbsp;

&nbsp;

In a few hours (minutes if you&#39;re fast), you&rsquo;ll get your first model trained and will be able to use your custom OCR API for parsing US Employment Authorization in your application.

&nbsp;

To get more information about the training phase, please refer to the getting started tutorial. If you have any questions regarding your use case, feel free to reach out!

&nbsp;

How to build an accurate US Employment Authorization OCR using Mindee's deep learning engine.

US Employment Authorization

<h1>Medicare health insurance card OCR</h1>

&nbsp;

This article explains how to quickly build and deploy an accurate OCR API that extracts data from US Medicare health insurance cards&nbsp;using our deep learning engine. It can be adapted to any type of health insurance card from other providers.

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 Health insurance card&nbsp;images or pdfs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

&nbsp;

<h2>Define your Health Insurance card data model</h2>

&nbsp;

First, we&rsquo;re going to define what fields we want to extract from your health insurance cards.

&nbsp;

<img alt="Health insurance card OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-54.png" />

&nbsp;

&nbsp;

Here is the field list we want our algorithm to look for in your documents:

&nbsp;

<ul>
	<li>Card owner: The full name card owner</li>
	<li>ID Number: The unique identifier claim number</li>
	<li>Sex: Gender of the card owner</li>
	<li>Hospital effective date: Start date of the hospital coverage&nbsp;</li>
	<li>Medical effective date: Start date of the medical coverage</li>
</ul>

&nbsp;

If you&#39;re following this article for other types of health insurance cards, feel free to add as many as you want.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined the field list, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page.&nbsp;<a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-52.png" target="_blank">Here is the image</a>&nbsp;you can use for setting up the API, and my setup looks like this:

&nbsp;

<img alt="Medicare health insurance ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-55.png" />

&nbsp;

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. You land on a new page where we are going to define more technically the required field we want to extract in our new OCR API.

 
&nbsp;

To go further, you can either&nbsp;<a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/us_health_insurance_card-config" target="_blank">download this json config</a>&nbsp;to set up your data model (there is an upload button on the left part of the screen), or you can do it manually. Here is the config:

&nbsp;

Cardholder: type String. The cardholder name never contains numeric characters, you can specify this by clicking the corresponding checkbox

&nbsp;

<img alt="Cardholder health insurance ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-56.png" />

&nbsp;

&nbsp;

ID Number: type String. It&#39;s composed of both alpha and numeric characters so we don&#39;t put any specification for the claim number.

&nbsp;

<img alt="Claim number health insurance ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-57.png" />

&nbsp;

&nbsp;

Sex: type String. Like cardholder, there are no numeric characters for this one.

&nbsp;

<img alt="Gender field for health insurance ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-58.png" />

&nbsp;

&nbsp;

Hospital effective date: type Date

&nbsp;

<img alt="Hospital effective date health insurance ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-59.png" />

&nbsp;

&nbsp;

Medical&nbsp; effective date: type Date

&nbsp;

<img alt="Medical effictive date health insurance OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-60.png" />

&nbsp;

&nbsp;

<h2>Train your health insurance card model</h2>

&nbsp;

<img src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image.png" />

&nbsp;

Your setup is done. You can now deploy your API and train your model for automatically extracting data from your health insurance cards using the API.

&nbsp;

To get more information about the training phase, please refer to the&nbsp;<a href="https://mindee.com/documentation/beta-api-builder/getting-started">getting started tutorial</a>. If you have any questions regarding your use case, feel free to reach out!

Medicare health insurance card

<h1>French health insurance card OCR 
Carte mutuelle</h1>

&nbsp;

This article explains how to extract all the relevant data from French health insurance cards (carte mutuelle tiers payant) using our deep learning OCR engine.

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account.&nbsp;<a href="https://beta.mindee.net/">Sign up</a>&nbsp;and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 Health insurance card&nbsp;images or pdfs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

&nbsp;

<h2>Define the data you need from Health insurance cards</h2>

&nbsp;

We need first to set up our use case by defining&nbsp;what fields we want to extract from your health insurance cards.

&nbsp;

&nbsp;

<img alt="Health insurance card (Carte mutuelle) OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-61.png" />

&nbsp;

&nbsp;

In this article, we are going to set up an API for extracting the following fields:

&nbsp;

<ul>
	<li>AMC Number: The AMC number of the cardholder</li>
	<li>Member identification number: The unique member identification number (num&eacute;ro adh&eacute;rent)</li>
	<li>Teletransmission number</li>
	<li>Insured full name:&nbsp;Full name of the first insured. You can add more fields of this type if you want to extract more of them.</li>
	<li>Insured social security number: SSN of the first insured.</li>
	<li>Validity start date</li>
	<li>Validity end date</li>
</ul>

&nbsp;

You can add as many fields as you want to fit your requirements.

&nbsp;

&nbsp;

<h2>Deploy your Health Insurance card API</h2>

&nbsp;

The field list was defined and we are now going to set up our API. Head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page.&nbsp;<a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/tiers-payant-recto-jpg.png" target="_blank">Here is the image</a>&nbsp;you can use, and the setup looks like this:

&nbsp;

<img alt="Setup health insurance card OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-63.png" />

&nbsp;

Click on the &ldquo;next step&rdquo; button. You land on a new page where we are going to add the technical definitions of the fields we defined above.

&nbsp;

To go further, you can either&nbsp;<a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/french_insurance_card-config" target="_blank">download this json config</a>&nbsp;to set up your data model (there is an upload button on the left part of the screen), or you can do it manually. Here is the config:

&nbsp;

&nbsp;

AMC Number: type String with no alpha characters

&nbsp;

<img alt="AMC number health insurance card ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-70.png" />

&nbsp;

&nbsp;

Member identification number: type String with no alpha characters

&nbsp;

<img alt="Member identifier health insurance card ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-69.png" />

&nbsp;

&nbsp;

Teletransmission number: type String with no alpha characters

&nbsp;

<img alt="Teletransmission number health insurance card ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-68.png" />

&nbsp;

&nbsp;

Insured full name:&nbsp;type String with no numeric characters

&nbsp;

<img alt="Full name health insurance card ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-67.png" />

&nbsp;

&nbsp;

Insured social security number: type String without any specification

&nbsp;

<img alt="ssn health insurance card ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-66.png" />

&nbsp;

&nbsp;

Validity start date: type Date

&nbsp;

<img alt="validity start date health insurance card ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-65.png" />

&nbsp;

&nbsp;

Validity end date: type Date

&nbsp;

<img alt="Validity end date health insurance card ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-64.png" />

&nbsp;

&nbsp;

That&#39;s it for the setup phase. Let&#39;s deploy your OCR.

&nbsp;

<h2>&nbsp;</h2>

<h2>Train your health insurance card model</h2>

&nbsp;

<img alt="Train health insurance card ocr" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/image-62.png" />

&nbsp;

&nbsp;

Your setup is done. You can now deploy your API and train your deep learning OCR to&nbsp;extract data from your health insurance cards using the API.

&nbsp;

To get more information about the training phase, please refer to the&nbsp;<a href="https://mindee.com/documentation/beta-api-builder/getting-started">getting started tutorial</a>. If you have any questions regarding your use case, feel free to reach out!

How to build an accurate French Health insurance card (cartes mutuelle) OCR using Mindee's deep learning engine.

French health insurance card

<h1>Blood Test Results OCR</h1>

&nbsp;

This article walks you through the building process of&nbsp;an OCR API that extracts data from Blood Test Results using our deep learning engine. It will work for any document issued by any laboratory.&nbsp;

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 Blood Test Results images or pdfs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your Blood Test Results use case</h2>

&nbsp;

First, we&rsquo;re going to define what fields we want to extract from your Blood Test Results.&nbsp;

&nbsp;

&nbsp;

<img alt="Blood Test Results OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/q.png" />

&nbsp;

&nbsp;

<ul>
	<li>Patient name: The full name of the patient&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Date of Report: The date of the report creation</li>
</ul>

&nbsp;

<ul>
	<li>ID Number: The patient ID Number</li>
</ul>

&nbsp;

<ul>
	<li>Sodium Results: The amount of Sodium the patient has in his organism according to the blood test results.&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Calcium Results: The amount of Calcium the patient has in his organism according to the blood test results.&nbsp;</li>
</ul>

&nbsp;

&nbsp;

That&rsquo;s it for our use case. Feel free to add any other relevant data to fit your requirements.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined the list of fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page. <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/blood-test.jpg">Here is the image </a>you can use for setting up the API, and my setup looks like this:

&nbsp;

&nbsp;

<img alt="Set up your Blood Test Results OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/r.png" />

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract.

 
&nbsp;

To go further, you can <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/blood_test_result-config.json">download this json config</a> to set up your data model or do it manually.

 
&nbsp;

Patient name: type String that never contains numeric characters.

&nbsp;

<img alt="Patient Name field for Blood Test Results OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/s.png" />

&nbsp;

&nbsp;

&nbsp;

Date of Report: type Date with US format.&nbsp;&nbsp;

&nbsp;

<img alt="Date field for Blood Test Results OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/t.png" />

&nbsp;

ID Number: type Number without specifications.

&nbsp;

<img alt="ID Number field for Blood Test Results OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/u.png" />

&nbsp;

Sodium Results:&nbsp;type Number without specifications.&nbsp;

&nbsp;

<img alt="Sodium Results field for Blood Test Results OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/v.png" />

&nbsp;

Calcium Results: type Number without specifications.&nbsp;

&nbsp;

<img alt="Calcium Results field for Blood Test Results OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/w.png" />

&nbsp;

&nbsp;

Once you&rsquo;re done setting up your data model, press the Data model ready? start training button at the top of the screen.

&nbsp;

&nbsp;

<h2>Train your Blood Test Results OCR</h2>

&nbsp;

&nbsp;

<img alt="Deep learning Blood Test Results OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/x.png" />

&nbsp;

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

Now is the time to train your Blood Test Results, deep learning model, in the Training section of our API.&nbsp;

&nbsp;

&nbsp;

In a few hours (minutes if you&#39;re fast), you&rsquo;ll get your first model trained and will be able to use your custom OCR API for parsing Blood Test Results in your application.

&nbsp;

To get more information about the training phase, please refer to the getting started tutorial. If you have any questions regarding your use case, feel free to reach out!

How to build an accurate Blood Test Results OCR using Mindee's deep learning engine.

Blood Test Results

<h1>Medical Certificate OCR</h1>

&nbsp;

This article walks you through the building process of&nbsp;an OCR API that extracts data from Medical Certificates using our deep learning engine.&nbsp;

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 Medical Certificate images or pdfs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your Medical Certificate use case</h2>

&nbsp;

First, we&rsquo;re going to define what fields we want to extract from your Medical Certificate.&nbsp;

&nbsp;

&nbsp;

<img alt="Medical Certificate OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/aaaa.png" />

&nbsp;

&nbsp;

<ul>
	<li>Student Name:&nbsp;The name of the certificate holder</li>
</ul>

&nbsp;

<ul>
	<li>Student Number: The student identification number&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Student School: The student&#39;s school for which the certificate was issued</li>
</ul>

&nbsp;

<ul>
	<li>Address: The medical examination center address&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Medical center name: The name of the medical center where the examination was performed</li>
</ul>

&nbsp;

&nbsp;

That&rsquo;s it for our use case. Feel free to add any other relevant data to fit your requirements.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined the list of fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page. <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/medical-certificate.jpg">Here is the image </a>you can use for setting up the API, and my setup looks like this:

&nbsp;

&nbsp;

<img alt="Set up your Medical Certificate OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/bbbb.png" />

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract.

 
&nbsp;

To go further, you can <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/medical_certificate-config.json">download this json config</a> to set up your data model or do it manually.

 
&nbsp;

Student Name:&nbsp;type String that never contains numeric characters.

&nbsp;

<img alt="Student Name field for Medical Certificate OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/cccc.png" />

&nbsp;

Student Number: type Number without specifications.&nbsp;&nbsp;

&nbsp;

<img alt="Student Number field for Medical Certificate OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/dddd.png" />

&nbsp;

Student School: type String that never contains alpha characters.&nbsp;

&nbsp;

<img alt="Student School field for Medical Certificate OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/eeee.png" />

&nbsp;

Address: type String without specifications.&nbsp;

&nbsp;

<img alt="Address field for Medical Certificate OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/ffff.png" />

&nbsp;

Medical Center: type String that never contains alpha characters.&nbsp;

&nbsp;

<img alt="Medical Center field for Medical Certificate OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/gggg.png" />

&nbsp;

&nbsp;

Once you&rsquo;re done setting up your data model, press the Data model ready? start training button at the top of the screen.

&nbsp;

&nbsp;

<h2>Train your Medical Certificate&nbsp;OCR</h2>

&nbsp;

&nbsp;

<img alt="Deep learning Medical Certificate OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/hhhh.png" />

&nbsp;

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

Now is the time to train your Medical Certificate deep learning model in the Training section of our API.&nbsp;

&nbsp;

In a few hours (minutes if you&#39;re fast), you&rsquo;ll get your first model trained and will be able to use your custom OCR API for parsing Medical Certificate&nbsp;in your application.

&nbsp;

To get more information about the training phase, please refer to the getting started tutorial. If you have any questions regarding your use case, feel free to reach out!

How to build an accurate Medical Certificate OCR using Mindee's deep learning engine.

Medical Certificate

<h1>Medical prescription OCR</h1>

&nbsp;

Reading this article you&#39;ll be able to build an OCR API that extracts data from medical prescription&nbsp;using our deep learning engine.

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need&nbsp;to <a href="https://beta.mindee.net/">sign up</a> to get a free beta account.&nbsp;</li>
	<li>You&rsquo;ll need at least 20&nbsp;different medical prescription&nbsp;images or pdfs to train your OCR model</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your medical prescription&nbsp;use case</h2>

<h2> 
&nbsp;</h2>

First, we&rsquo;re going to define what fields we want to extract from your medical prescription.

Here is the list of fields we are going to extract using our OCR API:

&nbsp;

<ul>
	<li>Date</li>
	<li>Doctor&#39;s name&nbsp;</li>
	<li>Patient&#39;s name</li>
	<li>Address: Of the medical center</li>
	<li>Phone number: Of the medical center</li>
</ul>

&nbsp;

&nbsp;

<img alt="Medical prescription OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/19/capture-decran-2021-01-19-a-102536.png" />

&nbsp;

&nbsp;

That&rsquo;s it for our use case. Feel free to add any other relevant data to fit your requirements.

&nbsp;

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

<h2>&nbsp;</h2>

Once you have defined what fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

 
&nbsp; 
You land now on the setup page. You can use any image you want for setting up the API, my setup looks like this:

&nbsp;

&nbsp;

 
<img alt="Setup your medical prescription OCR API”" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/capture-decran-2021-01-18-a-170724.png" />

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract.

&nbsp;

 
To go further, you can download<a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/19/medical_prescription-config-1.json"> this json config</a> to set up your data model or do it manually.

&nbsp;

Date:&nbsp;in European format

&nbsp;<img alt="Date field for medical prescription OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/capture-decran-2021-01-18-a-171028.png" />

Doctor&#39;s name: A name never contains numeric characters

<img alt="doctor name field for medical prescription OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/capture-decran-2021-01-18-a-171346.png" />

&nbsp;

Patient&#39;s name:&nbsp;A name never contains numeric characters

<img alt="patient name field for medical prescription OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/capture-decran-2021-01-18-a-172118.png" />

&nbsp;

&nbsp;

Medical center&#39;s address:&nbsp;an address may contain both numeric and alpha characters

<img alt="address field for medical prescription OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/capture-decran-2021-01-18-a-171354.png" />

&nbsp;

&nbsp;

Medical center&#39;s phone number:&nbsp;

<img alt="Phone number field for medical prescription OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/capture-decran-2021-01-18-a-171536.png" />

Once you&rsquo;re done setting up your data model, press the Data model ready? start training button at the top of the screen.

&nbsp;

&nbsp;

&nbsp;

<h2>Train your medical prescription&nbsp;OCR</h2>

&nbsp;

&nbsp;

<h2><img alt="Deep learning medical prescription OCR training" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/capture-decran-2021-01-18-a-172731.png" /></h2>

&nbsp;

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

&nbsp;

Now is the time to train your medical prescription deep learning model in the Training section of our API.&nbsp;You will soon be able to use your custom OCR API for medical prescriptions! When you will reach at least 20 documents to train your model you will receive a confirmation email allowing your model to make predictions.&nbsp;

&nbsp;

&nbsp;

To get more information about the training phase, please refer to the <a href="https://mindee.com/documentation/beta-api-builder/getting-started">getting started tutorial</a>. If you have any questions regarding your use case, feel free to reach out!

How to build an accurate Medical prescription OCR using Mindee's deep learning engine.

Medical prescription

<h1>Birth Certificate OCR</h1>

&nbsp;

This article walks you through the building process of&nbsp;an OCR API that extracts data from Birth certificates using our deep learning engine. It will work for any document issued in the United States.&nbsp;&nbsp;

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 Birth Certificate&nbsp;images or pdfs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your Birth Certificate use case</h2>

&nbsp;

First, we&rsquo;re going to define what fields we want to extract from your Birth Certificate.&nbsp;

&nbsp;

&nbsp;

<img alt="Birth Certificate OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/i.png" />

<ul>
	<li>First name: The first name of the birth certificate holder&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Last Name: The last name&nbsp;of the birth certificate holder</li>
</ul>

&nbsp;

<ul>
	<li>Birth Date: The date of birth of the birth certificate holder</li>
</ul>

&nbsp;

<ul>
	<li>City of Birth: The city of birth of the birth certificate holder</li>
</ul>

&nbsp;

<ul>
	<li>Certificate Number: The certificate Number&nbsp;</li>
</ul>

&nbsp;

&nbsp;

That&rsquo;s it for our use case. Feel free to add any other relevant data to fit your requirements.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined the list of fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page. <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/birth-certificate.png">Here is the image </a>you can use for setting up the API, and my setup looks like this:

&nbsp;

&nbsp;

<img alt="Set up your Birth Certificate OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/j.png" />

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract.

 
&nbsp;

To go further, you can <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/birth_certificate-config.json">download this json config</a> to set up your data model or do it manually.

 
&nbsp;

First name: type String that never contains numeric characters.

&nbsp;

<img alt="First Name field for Birth Certificate OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/k.png" />

&nbsp;

&nbsp;

&nbsp;

Last Name: type String&nbsp;that never contains numeric characters.&nbsp;

&nbsp;

<img alt="Last Name field for Birth Certificate OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/l.png" />

&nbsp;

Birth Date: type Date with US format.&nbsp;

&nbsp;

<img alt="Birth Date field for Birth Certificate OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/m.png" />

&nbsp;

City of Birth:&nbsp;type String that never contains&nbsp;numeric characters.&nbsp;

&nbsp;

<img alt="City of birth field for Birth Certificate OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/n.png" />

&nbsp;

Certificate Number: type Number without specifications.&nbsp;

&nbsp;

<img alt="Certificate Number field for Birth Certificate OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/o.png" />

&nbsp;

&nbsp;

Once you&rsquo;re done setting up your data model, press the Data model ready? start training button at the top of the screen.

&nbsp;

&nbsp;

<h2>Train your Birth Certificate&nbsp;OCR</h2>

&nbsp;

&nbsp;

<img alt="Deep learning Birth Certificate OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/p.png" />

&nbsp;

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

Now is the time to train your Birth Certificate deep learning model in the Training section of our API.&nbsp;

&nbsp;

&nbsp;

In a few hours (minutes if you&#39;re fast), you&rsquo;ll get your first model trained and will be able to use your custom OCR API for parsing Birth Certificate in your application.

&nbsp;

To get more information about the training phase, please refer to the getting started tutorial. If you have any questions regarding your use case, feel free to reach out!

How to build an accurate Birth certificates OCR using Mindee's deep learning engine.

Birth certificates

<h1>Marriage Certificate OCR</h1>

&nbsp;

This article walks you through the building process of&nbsp;an OCR API that extracts data from Marriage Certificates using our deep learning engine. It will work for any bank company or bank statement&nbsp;template.&nbsp;

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 Marriage Certificate images or pdfs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your Marriage Certificate use case</h2>

&nbsp;

First, we&rsquo;re going to define what fields we want to extract from your Marriage Certificate.&nbsp;

&nbsp;

&nbsp;

<img alt="Marriage Certificate OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/ttt.png" />

&nbsp;

&nbsp;

<ul>
	<li>Issuer Name:&nbsp;The name of the marriage certificate issuer</li>
</ul>

&nbsp;

<ul>
	<li>Issued date: The date of issue of the marriage certificate&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Husband&#39;s Name: The husband&#39;s name</li>
</ul>

&nbsp;

<ul>
	<li>Wife&#39;s name: The wife&#39;s name</li>
</ul>

&nbsp;

<ul>
	<li>Clerk information: The clerk address information</li>
</ul>

&nbsp;

&nbsp;

That&rsquo;s it for our use case. Feel free to add any other relevant data to fit your requirements.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined the list of fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page. <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/marriage-certificate.jpg">Here is the image </a>you can use for setting up the API, and my setup looks like this:

&nbsp;

&nbsp;

<img alt="Set up your Marriage Certificate OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/1.png" />

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract.

 
&nbsp;

To go further, you can <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/marriage_certificate-config.json">download this json config</a> to set up your data model or do it manually.

 
&nbsp;

Issuer Name:&nbsp;type String that never contains numeric characters.

<img alt="" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/vvv.png" />

&nbsp;

&nbsp;

&nbsp;

Issued Date: type Date with US format.&nbsp;&nbsp;

&nbsp;

<img alt="Issued date field for Marriage Certificate OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/3.png" />

&nbsp;

Husband&#39;s Name: type String that never contains numeric characters.&nbsp;

&nbsp;

<img alt="Account Number field for Electricity Bill OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/xxx.png" />

&nbsp;

Wife&#39;s name:&nbsp;type String that never contains numeric characters.&nbsp;

&nbsp;

<img alt="Bill date field for Electricity Bill OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/yyy.png" />

&nbsp;

Clerk Information: type String that never contains numeric characters.&nbsp;

&nbsp;

&nbsp;

<img alt="Amount Due field for Electricity Bill OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/zzz.png" />

&nbsp;

&nbsp;

Once you&rsquo;re done setting up your data model, press the Data model ready? start training button at the top of the screen.

&nbsp;

&nbsp;

<h2>Train your Marriage Certificate&nbsp;OCR</h2>

&nbsp;

&nbsp;

<img alt="Deep learning Electricity Bill OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/zzz2.png" />

&nbsp;

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

Now is the time to train your Marriage Certificate deep learning model in the Training section of our API.&nbsp;

&nbsp;

&nbsp;

In a few hours (minutes if you&#39;re fast), you&rsquo;ll get your first model trained and will be able to use your custom OCR API for parsing Marriage Certificate&nbsp;in your application.

&nbsp;

To get more information about the training phase, please refer to the getting started tutorial. If you have any questions regarding your use case, feel free to reach out!

How to build an accurate Marriage Certificate OCR using Mindee's deep learning engine.

Marriage Certificate

<h1>Diploma OCR</h1>

&nbsp;

Reading this article you&#39;ll be able to build an OCR API that extracts data from diploma&nbsp;using our deep learning engine.

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need&nbsp;to <a href="https://beta.mindee.net/">sign up</a> to get a free beta account.&nbsp;</li>
	<li>You&rsquo;ll need at least 20&nbsp;different diploma images or pdfs to train your OCR model</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your diploma prescription&nbsp;use case 
&nbsp;</h2>

First, we&rsquo;re going to define what fields we want to extract from your diploma.

&nbsp;

Here is the list of fields we are going to extract using our OCR API:

&nbsp;

<ul>
	<li>Date</li>
	<li>University name: String which contains both numeric and alpha characters&nbsp;</li>
	<li>Graduate name:&nbsp;String which&nbsp;Only contains alpha characters</li>
	<li>Major:&nbsp;String which only contains alpha characters</li>
	<li>Diploma number: Amount</li>
</ul>

&nbsp;

<img alt="Diploma OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/19/capture-decran-2021-01-19-a-160923.png" />

&nbsp;

&nbsp;

That&rsquo;s it for our use case. Feel free to add any other relevant data to fit your requirements.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

<h2>&nbsp;</h2>

Once you have defined what fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

 
&nbsp; 
You land now on the setup page. You can use any image you want for setting up the API, my setup looks like this:

&nbsp;

&nbsp;

&nbsp;

<img alt="Setup your diploma OCR API”" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/19/capture-decran-2021-01-19-a-145859.png" /> 
&nbsp;

&nbsp;

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract.

&nbsp;

 
To go further, you can download this <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/19/diploma-config.json">json config</a> to set up your data model or do it manually.

&nbsp;

&nbsp;

Date:&nbsp;in European format

&nbsp;<img alt="Date field for medical prescription OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/18/capture-decran-2021-01-18-a-171028.png" />

University name: String which contains both numeric and alpha characters

<img alt="University name for diploma OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/19/capture-decran-2021-01-19-a-145632.png" />

&nbsp;

Graduate name:&nbsp;A name never contains numeric characters

<img alt="" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/19/capture-decran-2021-01-19-a-145637.png" />

&nbsp;

Major:&nbsp;String which only contains alpha characters

&nbsp;

<img alt="Major field for diploma OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/19/capture-decran-2021-01-19-a-145644.png" />

&nbsp;

Diploma number:&nbsp;Amount

&nbsp;

<img alt="Diploma number for diploma OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/19/capture-decran-2021-01-19-a-145652.png" />

&nbsp;

&nbsp;

Once you&rsquo;re done setting up your data model, press the Data model ready? start training button at the top of the screen.

&nbsp;

&nbsp;

<h2>Train your diploma OCR</h2>

&nbsp;

<h2><img alt="Deep learning diploma OCR training" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/19/capture-decran-2021-01-19-a-160751.png" /></h2>

&nbsp;

&nbsp;

Now is the time to train your diploma deep learning model in the Training section of our API.&nbsp;You will soon be able to use your custom OCR API for diplomas! When you will reach at least 20 documents to train your model you will receive a confirmation email allowing your model to make predictions.&nbsp;

&nbsp;

&nbsp;

To get more information about the training phase, please refer to the <a href="https://mindee.com/documentation/beta-api-builder/getting-started">getting started tutorial</a>. If you have any questions regarding your use case, feel free to reach out!

How to build an accurate Diploma OCR using Mindee's deep learning engine.

Diploma

<h1>Certificate of Origin OCR</h1>

&nbsp;

This article walks you through the building process of&nbsp;an OCR API that extracts data from Certificate of Origin using our deep learning engine. It will work for any bank company or bank statement&nbsp;template.&nbsp;

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 Certificate of Origin&nbsp;images or pdfs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your Certificate of Origin use case</h2>

&nbsp;

First, we&rsquo;re going to define what fields we want to extract from your Certificate of Origin.&nbsp;

&nbsp;

&nbsp;

<img alt="Certificate of Origin OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/aa.png" />

<ul>
	<li>Agent Name: The full name of the agent&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Name and Address of shipper: The full name and address of the shipper&nbsp;&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Name and Address of consignee: The full name and address of the consignee&nbsp;&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Package Weight: The total package weight</li>
</ul>

&nbsp;

<ul>
	<li>Package Description: The package description</li>
</ul>

&nbsp;

&nbsp;

That&rsquo;s it for our use case. Feel free to add any other relevant data to fit your requirements.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined the list of fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page. <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/certificate-of-origin.jpg">Here is the image </a>you can use for setting up the API, and my setup looks like this:

&nbsp;

&nbsp;

<img alt="Set up your Certificate of Origin OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/bb.png" />

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract.

 
&nbsp;

To go further, you can <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/certificate_of_origin-config.json">download this json config</a> to set up your data model or do it manually.

 
&nbsp;

Agent name: type String that never contains numeric characters.

&nbsp;

<img alt="Agent Name field for Certificate of Origin OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/cc.png" />

&nbsp;

&nbsp;

&nbsp;

Name and Address of shipper: type String&nbsp;without specifications.&nbsp;

&nbsp;

<img alt="Name and Address of shipper field for Certificate of Origin OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/dd.png" />

&nbsp;

Name and Address of consignee: type String&nbsp;without specifications.&nbsp;

&nbsp;

<img alt="Name and address of consignee field for Certificate of Origin OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/ee.png" />

&nbsp;

Package weight:&nbsp;type Number without specifications.&nbsp;

&nbsp;

<img alt="Package Weight field for Certificate of Origin OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/ff.png" />

&nbsp;

Package Description: type String without specifications.&nbsp;

&nbsp;

<img alt="Package description field for Certificate of Origin OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/gg.png" />

&nbsp;

&nbsp;

Once you&rsquo;re done setting up your data model, press the Data model ready? start training button at the top of the screen.

&nbsp;

&nbsp;

<h2>Train your Certificate of Origin&nbsp;OCR</h2>

&nbsp;

&nbsp;

<img alt="Deep learning Certificate of Origin OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/hh.png" />

&nbsp;

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

Now is the time to train your Certificate of Origin deep learning model in the Training section of our API.&nbsp;

&nbsp;

In a few hours (minutes if you&#39;re fast), you&rsquo;ll get your first model trained and will be able to use your custom OCR API for parsing Certificate of Origin in your application.

&nbsp;

To get more information about the training phase, please refer to the getting started tutorial. If you have any questions regarding your use case, feel free to reach out!

How to build an accurate Certificate of Origin OCR using Mindee's deep learning engine.

Certificate of Origin

<h1>Certificate of Status OCR</h1>

&nbsp;

This article walks you through the building process of&nbsp;an OCR API that extracts data from Certificate of Status using our deep learning engine. It will work for any bank company or bank statement&nbsp;template.&nbsp;

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 Certificate of Status images or pdfs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your Certificate of Status use case</h2>

&nbsp;

First, we&rsquo;re going to define what fields we want to extract from your Certificate of Status.&nbsp;

&nbsp;

&nbsp;

<img alt=" Certificate of Status OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/ii.png" />

&nbsp;

<ul>
	<li>Entity name: The full name of the entity&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>File Number: The file number of your certificate of status</li>
</ul>

&nbsp;

<ul>
	<li>Formation date: The creation date of your entity</li>
</ul>

&nbsp;

<ul>
	<li>Type: The type of entity</li>
</ul>

&nbsp;

<ul>
	<li>Jurisdiction: The jurisdiction that your entity depends on</li>
</ul>

&nbsp;

<ul>
	<li>Status: The status of your entity</li>
</ul>

&nbsp;

&nbsp;

That&rsquo;s it for our use case. Feel free to add any other relevant data to fit your requirements.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined the list of fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page. <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/certificate-of-status.jpg">Here is the image </a>you can use for setting up the API, and my setup looks like this:

&nbsp;

&nbsp;

<img alt="Set up your Certificate of Status OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/jj.png" />

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract.

 
&nbsp;

To go further, you can <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/certificate_of_status-config.json">download this json config</a> to set up your data model or do it manually.

 
&nbsp;

Entity name: type String that never contains numeric characters.

&nbsp;

<img alt="Entity Name field for Certificate of Status OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/kk.png" />

&nbsp;

&nbsp;

&nbsp;

File Number: type Number without specifications.&nbsp;

&nbsp;

<img alt="File Number field for Certificate of Status OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/ll.png" />

&nbsp;

Formation Date: type Date with US format.

&nbsp;

<img alt="Formation Date field for Certificate of Status OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/mm.png" />

&nbsp;

Type:&nbsp;type String that never contains numeric characters.&nbsp;

&nbsp;

<img alt="Type field for Certificate of Status OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/nn.png" />

&nbsp;

Jurisdiction: type String that never contains numeric characters.&nbsp;

&nbsp;

<img alt="Jurisdiction field for Certificate of Status OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/oo.png" />

&nbsp;

&nbsp;

Status: type String that never contains numeric characters.&nbsp;

&nbsp;

<img alt="Status field for Certificate of Status OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/pp.png" />

&nbsp;

&nbsp;

Once you&rsquo;re done setting up your data model, press the Data model ready? start training button at the top of the screen.

&nbsp;

&nbsp;

<h2>Train your Certificate of Status OCR</h2>

&nbsp;

&nbsp;

<img alt="Deep learning Certificate of Status OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/qq.png" />

&nbsp;

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

Now is the time to train your Certificate of Status deep learning model in the Training section of our API.&nbsp;

&nbsp;

In a few hours (minutes if you&#39;re fast), you&rsquo;ll get your first model trained and will be able to use your custom OCR API for parsing Certificate of Status in your application.

&nbsp;

To get more information about the training phase, please refer to the getting started tutorial. If you have any questions regarding your use case, feel free to reach out!

How to build an accurate Certificate of Status OCR using Mindee's deep learning engine.

Certificate of Status

<h1>Shipper&#39;s Letter of Instruction OCR</h1>

&nbsp;

This article walks you through the building process of&nbsp;an OCR API that extracts data from Shipper&#39;s letter of Instruction using our deep learning engine. This API works with any letter template.&nbsp;

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 Shipper&#39;s letter of Instruction images or pdfs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your Shipper&#39;s letter of Instruction use case</h2>

&nbsp;

First, we&rsquo;re going to define what fields we want to extract from your Shipper&#39;s letter of Instruction.&nbsp;

&nbsp;

&nbsp;

<img alt="Shipper's letter of Instruction OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/ba.png" />

&nbsp;

<ul>
	<li>Shipper: The shipper&#39;s full name&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Consignee:&nbsp;The consignee&#39;s full name&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Forwarding Agent: The forwarding agent&#39;s full name&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Reference: The Shipper&#39;s letter of an Instruction reference number</li>
</ul>

&nbsp;

<ul>
	<li>Declared Value: The declared value of shipped goods</li>
</ul>

&nbsp;

<ul>
	<li>Date of issue: The issue date of&nbsp;Shipper&#39;s letter of Instruction</li>
</ul>

&nbsp;

&nbsp;

That&rsquo;s it for our use case. Feel free to add any other relevant data to fit your requirements.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined the list of fields you want to extract from your letter of instruction, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page. <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/shippers-letter-of-instruction.jpg">Here is the image</a> you can use for setting up the API, and my setup looks like this:

&nbsp;

&nbsp;

<img alt="Set up your Shipper's letter of Instruction OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/bb.png" />

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract.

 
&nbsp;

To go further, you can <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/shipper_s_letter_of_instruction-config.json">download this json config</a> to set up your data model or do it manually.

 
&nbsp;

Shipper: type String that never contains numeric characters.

&nbsp;

<img alt="Shipper for Shipper's letter of Instruction OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/bc.png" />

&nbsp;

&nbsp;

Consignee: type String that never contains numeric characters.

&nbsp;

<img alt="Consignee field for Shipper's letter of Instruction OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/bd.png" />

&nbsp;

Forwarding Agent: type String that never contains numeric characters.

&nbsp;

<img alt="Forwarding Agent field for Shipper's letter of Instruction OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/be.png" />

&nbsp;

Reference: type Number without specifications.&nbsp;

&nbsp;

<img alt="Reference field for Shipper's letter of Instruction OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/bf.png" />

&nbsp;

Declared Value: type Number without specifications.&nbsp;

&nbsp;

<img alt="Declared Value field for Shipper's letter of Instruction OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/bg.png" />

&nbsp;

Date of Issue: type Date with US format.&nbsp;

&nbsp;

<img alt="Date of issue field for Shipper's letter of instruction OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/bh.png" />

&nbsp;

Once you&rsquo;re done setting up your data model, press the Data model ready? start training button at the top of the screen.

&nbsp;

&nbsp;

<h2>Train your Shipper&#39;s letter of Instruction OCR</h2>

&nbsp;

&nbsp;

<img alt="Deep learning Shipper's letter of instruction OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/bi.png" />

&nbsp;

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

Now is the time to train your Shipper&#39;s letter of Instruction deep learning model in the Training section of our API.&nbsp;

&nbsp;

In a few hours (minutes if you&#39;re fast), you&rsquo;ll get your first model trained and will be able to use your custom OCR API for parsing Shipper&#39;s letter of Instruction in your application.

&nbsp;

To get more information about the training phase, please refer to the getting started tutorial. If you have any questions regarding your use case, feel free to reach out!

How to build an accurate Shipper's letter of Instruction OCR using Mindee's deep learning engine.

Shipper's letter of Instruction

<h1>Packing List OCR</h1>

&nbsp;

This article walks you through the building process of&nbsp;an OCR API that extracts data from Packing List forms using our deep learning engine.&nbsp;

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 Packing List images or pdfs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your Packing List use case</h2>

&nbsp;

First, we&rsquo;re going to define what fields we want to extract from your Packing List.&nbsp;

&nbsp;

&nbsp;

<img alt="Packing List OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/iiii.png" />

&nbsp;

&nbsp;

<ul>
	<li>Exporter:&nbsp;The name of the exporter</li>
</ul>

&nbsp;

<ul>
	<li>Exporter address: The exporter&#39;s address&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Consignee: The consignee&#39;s name</li>
</ul>

&nbsp;

<ul>
	<li>Consignee Address: The consignee&#39;s address where the package must be delivered</li>
</ul>

&nbsp;

<ul>
	<li>Consignee Phone Number: The consignee&#39;s phone number</li>
</ul>

&nbsp;

<ul>
	<li>Method of dispatch: how the package is carried (sea)</li>
</ul>

&nbsp;

<ul>
	<li>Export Invoice Date: Issue date of the export invoice</li>
</ul>

&nbsp;

<ul>
	<li>Export Invoice Number: Identification number of the export invoice</li>
</ul>

&nbsp;

<ul>
	<li>Country of Origin: where the package comes from&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Country of Final Destination: the package final destination&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Total Weight: total weight of package carried&nbsp;</li>
</ul>

&nbsp;

&nbsp;

That&rsquo;s it for our use case. Feel free to add any other relevant data to fit your requirements.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined the list of fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page. <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/packing-list.png">Here is the image </a>you can use for setting up the API, and my setup looks like this:

&nbsp;

&nbsp;

<img alt="Set up your Packing List OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/jjjj.png" />

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract.

 
&nbsp;

To go further, you can <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/packing_list-config.json">download this json config</a> to set up your data model or do it manually.

 
&nbsp;

Exporter Name:&nbsp;type String that never contains numeric characters.

&nbsp;

<img alt="Exporter Name field for Packing List OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/kkkk.png" />

&nbsp;

Exporter Address: type String without specifications.&nbsp;&nbsp;

&nbsp;

<img alt="Exporter address field for Packing List OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/llll.png" />

&nbsp;

Consignee Name: type String that never contains alpha characters.&nbsp;

&nbsp;

<img alt="Consignee name field for Packing List OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/mmmm.png" />

&nbsp;

Consignee Address: type String without specifications.&nbsp;

&nbsp;

<img alt="Consignee address for Packing List OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/nnnn.png" />

&nbsp;

Consignee phone number: type Phone Number.&nbsp;

&nbsp;

<img alt="Consignee phone number field for Packing List OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/oooo.png" />

&nbsp;

Method of Dispatch: type String that never contains alpha characters.&nbsp;

&nbsp;

<img alt="Method of Dispatch field for Packing List OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/pppp.png" />

&nbsp;

Export Invoice Date: type Date with US format.&nbsp;

&nbsp;

<img alt="Export invoice date field for Packing List OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/qqqq.png" />

&nbsp;

Export Invoice Number: type Number without specifications

&nbsp;

<img alt="Export invoice number field for Packing List OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/rrrr.png" />

&nbsp;

Country of Origin: type String that never contains numeric characters.&nbsp;

&nbsp;

<img alt="Country of origin field for Packing List OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/ssss.png" />

&nbsp;

Country of Final Destination: type String that never contains numeric characters.&nbsp;

&nbsp;

<img alt="Country of final Destination field for Packing List OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/tttt.png" />

&nbsp;

Total Weight: type Number without specifications.&nbsp;

&nbsp;

<img alt="Total Weight field for Packing List OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/uuuu.png" />

&nbsp;

&nbsp;

Once you&rsquo;re done setting up your data model, press the Data model ready? start training button at the top of the screen.

&nbsp;

&nbsp;

<h2>Train your Packing List&nbsp;OCR</h2>

&nbsp;

&nbsp;

<img alt="Deep learning Packing List OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/22/vvvv.png" />

&nbsp;

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

Now is the time to train your Packing List deep learning model in the Training section of our API.&nbsp;

&nbsp;

In a few hours (minutes if you&#39;re fast), you&rsquo;ll get your first model trained and will be able to use your custom OCR API for parsing Packing List&nbsp;in your application.

&nbsp;

To get more information about the training phase, please refer to the getting started tutorial. If you have any questions regarding your use case, feel free to reach out!

&nbsp;

How to build an accurate Packing List OCR using Mindee's deep learning engine.

Packing List

<h1>Delivery Note OCR</h1>

&nbsp;

This article walks you through the building process of&nbsp;an OCR API that extracts data from Delivery Note using our deep learning engine. It will work for any bank company or bank statement&nbsp;template.&nbsp;

&nbsp;

&nbsp;

<h2>Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free beta account. <a href="https://beta.mindee.net/">Sign up</a> and confirm your email to login.</li>
	<li>You&rsquo;ll need at least 20 Delivery Note images or pdfs to train your OCR.</li>
</ol>

&nbsp;

&nbsp;

<h2>Define your Delivery Note use case</h2>

&nbsp;

First, we&rsquo;re going to define what fields we want to extract from your Delivery Note.&nbsp;

&nbsp;

&nbsp;

<img alt="Delivery Note OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/rr.png" />

<ul>
	<li>Entity Name: The full name of the entity&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Code: The delivery code</li>
</ul>

&nbsp;

<ul>
	<li>Shipping Method: The shipping Method&#39;s code</li>
</ul>

&nbsp;

<ul>
	<li>Sales Person: The full name of the salesperson&nbsp;</li>
</ul>

&nbsp;

<ul>
	<li>Order Number: The order number</li>
</ul>

&nbsp;

&nbsp;

That&rsquo;s it for our use case. Feel free to add any other relevant data to fit your requirements.

&nbsp;

&nbsp;

<h2>Deploy your API</h2>

&nbsp;

Once you have defined the list of fields you want to extract, head over to the platform and press the &lsquo;build a new endpoint&rsquo; button.

&nbsp;

You land now on the setup page. <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/delivery-note.jpg">Here is the image </a>you can use for setting up the API, and my setup looks like this:

&nbsp;

&nbsp;

<img alt="Set up your Delivery Note OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/ss.png" />

&nbsp;

Once you&rsquo;re ready, click on the &ldquo;next step&rdquo; button. We are going to specify the data types for each of the fields we want our API to extract.

 
&nbsp;

To go further, you can <a href="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/delivery_note-config.json">download this json config</a> to set up your data model or do it manually.

 
&nbsp;

Entity name: type String that never contains numeric characters.

&nbsp;

<img alt="Entity Name field for Delivery Note OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/tt.png" />

&nbsp;

&nbsp;

&nbsp;

Code: type String&nbsp;without specifications.&nbsp;

&nbsp;

<img alt="Code field for Delivery Note OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/uu.png" />

&nbsp;

Shipping Method: type String that never contains numeric characters.&nbsp;

&nbsp;

<img alt="Shipping Method field for Delivery Note OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/vv.png" />

&nbsp;

Salesperson:&nbsp;type String that never contains numeric characters.&nbsp;&nbsp;

&nbsp;

<img alt="Sales Person field for Delivery Note OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/ww.png" />

&nbsp;

Order Number: type String without specifications.&nbsp;

&nbsp;

<img alt="Order Number field for Delivery Note OCR" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/xx.png" />

&nbsp;

&nbsp;

Once you&rsquo;re done setting up your data model, press the Data model ready? start training button at the top of the screen.

&nbsp;

&nbsp;

<h2>Train your Delivery Note OCR</h2>

&nbsp;

&nbsp;

<img alt="Deep learning Delivery Note OCR API" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2021/01/21/yy.png" />

&nbsp;

&nbsp;

You&rsquo;re all set!&nbsp;

&nbsp;

Now is the time to train your Delivery Note deep learning model in the Training section of our API.&nbsp;

&nbsp;

In a few hours (minutes if you&#39;re fast), you&rsquo;ll get your first model trained and will be able to use your custom OCR API for parsing Delivery Note in your application.

&nbsp;

To get more information about the training phase, please refer to the getting started tutorial. If you have any questions regarding your use case, feel free to reach out!

&nbsp;

<h1>&nbsp;</h1>

How to build an accurate Delivery Notes OCR using Mindee's deep learning engine.