A new SharePoint Premium prebuilt model called the Simple Document Processing model has been added. This model is promising as it’s pre-trained to work with structured documents and extract key-value pairs, checkboxes, named entities and barcodes. The idea is employees can quickly & easily use this model with a type of document and it will auto detect extractors which will then be extracted as metadata on the file in SharePoint. There is no advanced configuration or training of the model required!
The Simple Document Processing Model has been pre-trained to look for the following in a document:
- Key-value pairs – Think of these like labels and their corresponding information, such as “Name: Jamie Vardy.”
- Selection marks – These are checkboxes or other marks that indicate choices or selections in a document.
- Named entities – These are specific items like names of people, places, or organisations mentioned in the text of a document.
- Barcodes – These are machine-readable representations of data that can be used for tracking or identification purposes in a document.
Source: Microsoft Learn: Use a prebuilt model to extract information from simple documents in Microsoft Syntex
The model can currently only be used with PDF documents and images. So unfortunately Word documents (.docx) are currently not supported without them being converted to PDF – native Word document format support would be a big winner! Apparently more supported file types will be added in future releases. The model also also supports recognising extractor fields in over 100 languages.
Structured documents such as Forms, Invoices, Receipts, Contracts & Bank Statements work best with the Simple document processing model. You are typically looking for documents that have clear fields with corresponding labels to extract key-value pairs and works best with documents that contains well defined tables and sections. The documents are also processed for Optical Character Recognition (OCR) so the extracted text is also available in Search or for compliance reasons.
Setup a Simple Document Processing Model
Below I will walk you through the process of enabling the prebuilt Simple Document Processing model in your tenant, configure a model and add it to a library.
First ensure Syntex Pay As You Go. is setup and linked to an Azure Subscription this can be done in the M365 Admin Centre with the Syntex Admin Centre (Link).
In the M365 Syntex admin centre ensure your Azure subscription is setup and then browse to the Documents & images services section then click on Prebuilt document processing.
Here are the current configuration options to turn on and off prebuilt models. NOTE: there are currently five prebuilt models available (Receipts, Contracts, Invoices, Sensitive Information & Simple) and it would be great to turn on/off just specific prebuilt model types i.e. Receipts only rather than having to enable all.
You can then choose which sites to enable Prebuilt models for.
There are three options –All sites or Selected sites (up to 100) (search for site or upload a csv of site urls). I’m ok with Prebuilt being used on all sites so selected All sites then clicked Save.
That’s literally all the admin configuration needed & is possible at the current time! As mentioned previously it would be great if there was the ability to turn on/off specific prebuilt models rather than just a nuclear on/off and also the ability to programmatically enable sites for models using a API rather than CSV.
Configuring the Simple Document Processing Model
Go to your SharePoint Premium Content Centre site and go to the Create a model button. Then from the model creation screen click the Prebuilt models tab and then select Simple document processing.
Further details will then be provided about the model and an example file of trade securities is annotated with example extractors. Along with a link to the Microsoft site with the documentation for the Simple Document Processing Model.
I then name my Simple document processing model and click create.
The model configuration screen is then displayed to guide me through creating my first Simple Document Processing Model. I then need to add a file to use as the base document for my model. I will use this for the model to identify extractors and then I can select which extractors to include..
The Document is then processed using AI and OCR – the key value pairs and named entities are then identified and listed in Extractors found in this file section. The page range (to only process a certain range or page of a document) and the detected language can also be set here.
You can then review the identified extractors choose which extractors to use and rename them and change the column type (to be used in the SharePoint document library). NOTE: with the prebuilt model if the an extractor is not identified then there is no manual configuration to ensure that this value is also extracted.
Review the following identified extractors and then click Next.
Specify which libraries to add the new model to and click Add.
Now I have uploaded documents to the library you can see the files have been processed and most metadata values extracted from the documents. However there appears to be a bug perhaps in that it has not extracted all the fields i.e. unit price, total, subtotal, fees, execution, total settlement even though they were identified and selected as extractors when I configured/trained the model. These fields also occur in every document which is the same format/layout and contains the same fields so I would expect this model to work well (see below).
See below for an annotated version of my Trades Confirmation document in the Simple Document Processing Model I created. You can see it identified all the extractors and they were selected to be applied.
So it’s a bit puzzling why when I applied the model to my library and added documents (11 x Trade Confirmation PDFs in the same format with the same fields.) it didn’t extract all the fields. Hopefully this can be improved!
Seeing how other documents work with the simple model
I will now see how the model works with various different types of documents and review the extractors it found versus what I would expect it to find.
Purchase Order
Works well and you can see by the highlighted green sections of the extractors that it has identified most of the fields that I would possibly like to extract.
Most key value pairs are extracted denoted by a header i.e. SHIPPING TERMS and then the value underneath i.e. NET 60 is then extracted. Line items in the purchase order are also not extracted – this can be done with the SharePoint Premium Structured Document processing model. It looks like the model will only extract one Phone and Fax value so will not extract multiple values on the document if they are named the same.
Invoice
Invoice works similar to purchase order – key fields are extracted but line items are not extracted.
Contract
As previously mentioned the model doesn’t work well with documents with lots of text in paragraphs so it’s no surprise only 4 fields have been identified in a 7 page contract. The date which is ok, Client & contractor where it has extracted their name and address all as one line.
Technical Drawing (Plan)
I had great hopes with the simple document processing model and Drawing Plans (from an Architect) that it would be able to read the details block and extract the key value pairs i.e. Client, Project, Drawing Title, Revision but unfortunately i didn’t extract any of those and instead extracted tiny text from the actual drawings. It would be a killer feature to have a prebuilt model just for technical drawing plans!
Barcodes
Another area I see great potential and is a common ask from customers is the ability to scan barcodes in documents. So I created this test document with 4 different barcodes in Code128, EAN13m QRCode and UPCA format,
The model works really well in both identifying the format of the barcodes and extracting the underlying information contained within the barcode (see image below)!
I think the ability of the simple document processing model to extract barcodes is a big winner and I can see my customers using this. They often have cover sheets on documents with a barcode/QR code with further details.
BONUS- Detect the language of a Document (Super Cool!)
Also another feature that is really COOL and I dont think Microsoft have shouted about enough is the ability for the simple model to detect the language of a document and write it to a column.
I have clients doing this using Autofill columns where they have a generative AI prompt associated to a document library column i.e. “what is the language of this document output in ISO 639 3 letter code i.e. ENG”. The GPT4-Turbo model behind the scenes then reads the document and outputs to the column a three letter code for the language identified which works really well. However due to Autofill being relatively expensive at $0.05 per page they are finding it quite expensive for 100 page documents for example where each page is processed & charged. I then had an idea to use the new Simple model which has inbuilt Language detection and using the “Page range” functionality limit it to only the first page (see image below). So only the first page is processed and the cost is only $0.01.
A gotcha is at least one extractor has to be selected in order for the model to be able to be added to a library. I tried to save the model without having selected an extractor but I was then unable to add the model to any libraries. It would be great if in the future this model could be run using just the language detection feature and no extractors have to be selected.
Here is what the model looks like when added to a library and documents added. Note the new column added with the language code for the identified language of each document.
Summary
The Simple document processing model is a great idea and it’s great as an enabler for users to be quickly able to use AI to extract values from their documents without significant training & configuration. However, in my initial tests, when the model was added to libraries and documents were subsequently added, it missed configured extractors, resulting in missing fields in SharePoint, despite the fields being present in the documents. Nevertheless, it shows great promise, especially when tested and configured with other document types in the model training screen, where the correct extractors were identified. I’ll play around with some more document types to try and work out why the model is often not extracting values for configured extractors in some documents when docs are added to the published model.
I’ve been most impressed with language identification and barcode extraction at present – these will be super powerful for my customers!
How will you use this model?
