Automating Document Migration and File Management
SphereGen Case Study
A Connecticut Manufacturer needed to migrate their documents from an outdated document management system to SharePoint. The time required to complete this task manually was estimated to be over 2 years.
Learn how SphereGen was able to use RPA to automate this task, cutting the document migration process from 24+ months to 4 months. In addition, the automation was modified to handle new document filing and indexing, reducing file upload times by 90%.
Manufacturers are required by law to keep all documents related to their product sales, including materials components and line generation, for many years. Our manufacturing client used an older electronic document management system which was no longer supported. This system was not searchable as multiple documents were filed together in one scan.
Our client needed to correct this filing issue and set up a new management system which would correctly tag documents so they could be searched. SphereGen was able to build an RPA automation which handled the separation and migration of documents to SharePoint, as well as establish a front end to their SharePoint system which allows new documents to be indexed correctly as they are filed. The automation cut months off their document migration timeline and reduced new document uploading by 7-8 minutes per document.
Our client faced two challenges. Their document management system was not searchable as multiple documents were combined into one scan. As a result, they needed to migrate all documents to a new document management system. This meant breaking apart the scans into multiple documents and tagging them for search. In addition, they needed an easier way to load and tag new documents to their management system.
Manually breaking scans into individual documents for upload to a new management system was incredibly time consuming. SphereGen created an automation which read index markers on the old system, used those markers to identify a specific document, and then created a pdf of the individual document. The pdf is catalogued and loaded to SharePoint. Any new documents which must be filed are indexed and loaded into the proper SharePoint folder using a front-end application which incorporates Intelligent Document Processing to create searchable tags based on document data.
Automating the document migration reduced the migration timeline by almost 2 years.
In order to migrate and tag individual documents correctly, large .tiff and .pdf scans had to be broken into multiple pieces for each document. Trying to accomplish this manually presented an overwhelming task with a never-ending timeline.
Instead, the automated migration process was built to create new SharePoint folders, identify .tiff or .pdf documents in the old system and create new pdfs, that could be mapped and uploaded to the correct SharePoint folder creating searchable tags for each document.
After the migration was complete, our client still needed an efficient way to load and tag new documents so they could also be filed in the appropriate folder.
Building SharePoint Folder System
Before migration took place, the SharePoint Folder system was created. Using a Rules index which contains information for document types and mappable columns, an automation ran which created all necessary SharePoint folders according to the established rules. These rules included document name and all tagging information which needed to be assigned to a document that would reside in that folder.
This rules index is used throughout all other processes.
After the SharePoint folders were created, the migration automation handled the document conversion and upload to SharePoint.
The migration started by retrieving documents from the old system according to the indexes. These indexes contain the path of the document, the document type and what pages it resides on. The pages were an important piece of information as a document could be shared across many folders, with pages located in different folders. The migration bot used the document path and pages to read the old .tiff and .pdf files and select all pages of the document for conversion to a new pdf.
To correctly match the file to a SharePoint folder, the bot used the indexing information and the SharePoint rules to retrieve the appropriate folder name based on the document type. Upon a successful match, the file is correctly tagged according to the rules and uploaded to SharePoint.
The information for every transaction is written to a log. Any unsuccessful matches or files which could not be processed are recorded in the log with all pertinent information regarding the error and how to locate the file which did not process.
The migration was able to run 24/7 to migrate all files. The bots were monitored to make sure they continued to actively run.
New Document Management
To assist with new document filing, a front-end indexing automation was written. New pdfs are uploaded to a folder based on document type. The bot reads the folder, picks up new documents and loads them to the document management system using the rules-based logic from the migration. During this process, the bot uses Intelligent Document Processing to read the pdf and find the necessary field values required to assign searchable tags.
This required the training and testing of the bots to ensure that documents could be identified correctly.
Reduced Processing Time
The process of manually loading documents into the old document management system required 7-8 minutes per document. With the new automation, documents can be uploaded and correctly indexed and filed in less than 1 minute, improving processing time by 90%.
By automating the migration and filing of documents into SharePoint, our client was able to recognize time savings, improve productivity and gain functionality:
- Migration time – By automating the migration, the overall transfer of documents could be completed in a few months instead of over 2 years – an 84% improvement in time required by manual efforts.
- Productivity – When loading new documents to SharePoint, the upload and indexing time has been reduced from 8 minutes to 1, a 90% improvement.
- Better Functionality – The SharePoint document management system allows successful searching of needed documents by type and customer.
Automating the processing of pdfs is just one way you can use AI based RPA functions to streamline workflows and improve productivity. If you have any questions regarding RPA and its benefits, we would be happy to discuss any potential opportunities for automation you may have!
SphereGen is a technology company that specializes in developing innovative solutions to improve business outcomes. Our customer-centric approach focuses on finding the right solution to meet the need at hand, using technologies like Robotic Process Automation (RPA) to improve workflow and productivity. We also offer custom software services in Application Modernization/Support and Extended Reality.
As a privately-held small business, we understand the goals, needs, and challenges of organizations your size. SphereGen leverages 10+ years of experience to find the ‘best technology fit’ for clients whether you build, buy or customize software and services.