Bulk Document Generation with PowerAutomate and DocumentsCorePack

1369 views 0 Updated on November 7, 2022

This article describes step by step the best practice for bulk-document generation via DocumentsCorePack (DCP) in Power Automate (Microsoft Flow).

Sometimes it is required to generate a high number of documents in a short period of time. We recommend load-balancing if you require to generate a batch of documents. This will lead to a significant performance improvement, as can be seen in this article: “Performance benchmark, limitations & scalability“.

When building the flow to generate bulk documents, there are two goals:

  1. Create document jobs fast: In order for our services to start generating documents, the flow has to create the document jobs as fast as possible in an asynchronous manner. Thus the queue of documents to generate gets filled and the service(s) will be able to start generating the documents in bulk.
  2. If possible, combine document generation and processing settings into a single document job: DCP allows the combination of generation and processing steps into one document request. Thus, you can prevent having to set up additional actions in your flow to process the document.
    E.g. A single request can have both:
    1. Information on what document should be generated.
    2. information on how it should be processed (e.g. Save to SharePoint, Attach to email,…).

Keeping those two goals in mind, we will continue on and start setting up the flow:

1st Loop: Generate document requests

After loading the records, you need to generate the documents (e.g. via the DataVerse – List rows action) and generate your document requests via an Apply-to-Each loop.

Please note: You must loop through all records that you want to generate a document for. In this article, we are looping through 1000 account records.

Within your loop, you have 2 possibilities to create DocumentJobs.

1. Use the DocumentsCorePack Connector

The DCP connector has a Create Document Job (async) action to create the queue item:


Figure 1: DCP Connector

This action allows you to directly specify whether a document should be stored in SharePoint or be printed. If one is required, select it here to avoid additional requests later on.

If you plan to do the document processing inside your flow, you have to save the DocumentJobId in an array (see information on 2nd loop further on).

2. Use the DataVerse Connector:

You can use the Dataverse connector to create the queue items (add rows for mscrm-addons.com AutoMergeworkingitems). In general, it works the same. We need a loop to generate the queue items, but now we take the OOTB Dataverse Connector Add a row action.

Please Note: The setup is a bit more complex, but this is the fastest way to create queue items (DocumentJobs). 

Here you can see how the flow should look:

Figure 2: Sequence of the flow

There is an additional step to read the Template ID. We need this Template ID in the Dataverse – Add a row step to specify the template to be used. 

The rest of the loop is the same as above. Again, you need to store the AutoMergeWorkingItems Unique identifier which is the DocumentJobID to an array.

Additionally, you need to fill out the following fields in the Dataverse Action:

  • Template ID:
  • Account ID
  • File Type:

You can define additional document processing settings directly on this Item (e.g. Attach to and send an Email, Save to SharePoint, etc.). Again, the goal is to keep the number of requests to a minimum, so always try to process the document in the same Action.

What is the difference between the DocumentsCorePack Connector and the Dataverse Connector?

DocumentsCorePack Connector:

  • User-friendly & intuitive setup.
  • No need for a Get Template step, because the template can be directly specified in the Create Document Job (Async) step.

Dataverse Connector:

  • It allows you to generate, attach and send documents via email in a single Document Job.
  • It is the fastest way to create the items, since the Add a row action will not count towards the service protection API limit as outlined in this article.

Recommendation: if you have to email your documents or if you have to deal with very high volumes (10000+ records), the Dataverse connector is the better choice. For smaller batches, both options are fine with a negligible difference in performance.

2nd Loop: Get generated documents for processing

In order to process the finished documents in your flow, a second loop is required through the DocumentJobIds saved in our array.


Figure 3: Get DocumentsJob Result 2 (Preview)

The Get DocumentJob Result action provides the following information for every ID of our array:

  • File Name
  • File size
  • Document ( Base64 encoded)
  • Mime-Type
  • Success State
  • Return value

Thus, after retrieving this information you can add further steps to process the document as required by your business. You can use the full Power Automate suite to do so. Thus, the possibilities are endless. Some examples are:

  • Create an email and send the document as an attachment.
  • Save the document to OneDrive, DropBox, or Azure Blob.
  • Print the document.

Please note: If you want to speed up this loop, you can turn on the Concurrency Control to execute the loop parallel. If you do that, we recommend changing the retry policy as described below.

Troubleshooting

429 Error on one DocumentsCorePack Steps:


Figure 4: Error on DCP steps

If this error occurs, you are hitting an action limit from Microsoft Flow.

Please go to the settings of the one step:


Figure 5: Settings of one step

We recommend that you change the Retry Policy of this step to the following:


Figure 6: Settings

Back to “The DocumentsCorePack connector for PowerApps and PowerAutomate (Microsoft Flow)“.

That’s it! We appreciate your feedback! Please share your thoughts by sending an email to support@mscrm-addons.com

 

Was this helpful?

 
Related Articles