Intelligent Document Processing

In this, our first episode, Dr. Ryan Ries guides us through the changes to intelligent document processing and how major industries like healthcare, finance, and law, all will change because of the advent of generative AI.

In this episode, you will hear:

• How businesses are using intelligent document processing to drive efficiency
• Use cases for intelligent document processing
• What to do next if you're interested in IDP for your business

Show notes:

Learn if you're a fit for intelligent document processing on this use case page - https://aws.missioncloud.com/gen-ai-use-case-intelligent-document-processing 

Here's a blog where we discuss multiple different use cases for intelligent document processing - https://www.missioncloud.com/blog/generative-ai-use-cases-with-intelligent-document-processing

Here's AWS's own blog on how IDP is empowered by generative AI - https://aws.amazon.com/blogs/machine-learning/enhancing-aws-intelligent-document-processing-with-generative-ai/

Here's how AWS defines a foundation model- https://aws.amazon.com/what-is/foundation-models/

Check us out on the AWS Smart Business Hub - https://pages.awscloud.com/AMER-partner-GC-MissionCloud-IDP-2023-reg.html

Here's our marketplace offer for a free hour consulting on your IDP solution - https://aws.amazon.com/marketplace/pp/prodview-yllkfqeztqngi

If you just want to talk about Gen AI more generally, we also offer a free hour to consult on that - https://aws.amazon.com/marketplace/pp/prodview-cbpvma227mjym

Official Transcript

Intro

Welcome to Mission Generate, the podcast where we explore the power of AWS and the cutting-edge techniques we use with real customers to build generative AI solutions for them. Join us each episode as we dig into the technical details, cut through the hype, and uncover the business possibilities...

I'm Ryan Ries, our generative AI practice lead and your host.

In today's episode, we'll cover how to use intelligent document processing with Gen AI and how this is going to revolutionize several industries and maybe even change the meaning of paperwork?

Thanks for joining us, let's begin.

Episode 1

Okay, I have to come clean on something...

Remember when I said I was Ryan Ries? Well, I wasn't exactly being truthful...

You see, this is Ryan... well, it's my voice anyway.

But our senior product marketing manager, Casey, wrote me this script. And then he used my voice along with an AI to synthesize me speaking. See, most of the time, I'm busy meeting with customers and AWS. Far too busy to be hosting a podcast! But that doesn't mean I don't want to. Then we had an idea: what better way to cover Gen AI than to build a podcast that uses Gen AI to talk to you guys?

Cool, right? So you're listening to me, but in a way, you're really listening to a large language model... it's a philosophical distinction, I guess.

Hopefully we didn't freak you out too much. Ha ha... Okay, back to the topic at hand.

Let's talk IDP. Intelligent document processing is how to extract and organize data from various documents using the machine learning capabilities of AWS with tools like Textract and Comprehend. But there's an exciting new development in this field that takes IDP to the next level. It involves combining IDP with a large language model to synthesize and organize the information you're processing.

By merging these technologies into a unified solution, you can not only process structured data but also dive into unstructured data and generate meaningful outputs from it. This opens up a whole new world of possibilities...

So, what exactly is IDP? Well, it uses natural language processing to automate the extraction, interpretation, and processing of data from unstructured documents—similar techniques to how this podcast was made, actually. This eliminates the need for manual document processing, saving time and reducing errors. 

IDP goes beyond just digitizing documents, though. The labor of dealing with those documents, the inaccuracies that humans can introduce, the problems of scaling that kind of process... all of these disappear with an IDP-powered workflow. So you can begin to see how this could impact industries that deal with substantial paperwork: finance, insurance, healthcare, and law—all of these are going to change substantially in the coming decade because of IDP and because of the capabilities generative AI introduces.

Let's sketch this out in a practical example: using Amazon's Comprehend service, you could easily pull out key details from internal emails, such as entities, people, and keywords, to gain a better understanding of contract details. But you could then synthesize that data with a gen AI model to explain the various relationships and limitations delineated in a contract. Think about how that changes a legal team's ability to negotiate. Or the volume of business a firm can handle.

AWS, IDP, and generative AI are a perfect combination for this kind of work because AWS offers ready-to-use APIs for easy classification and extraction of critical data. More importantly, it also provides much greater control over security, compliance, and data privacy—this is not like handing your information over to ChatGPT and hoping your data remains secure. Now, sure, it can take more work to build a full solution on AWS, but this is how you ensure your data is never touching the public internet, for instance.

AWS offers powerful services like Textract and Comprehend which not only handle extraction at scale but will respect compliance directives for handling sensitive information, like medical records, for instance. You can even use them to locate and redact this information and meet data privacy laws before passing that data onto the next stage of your solution.

Let's also talk about foundation models for a minute. Foundation models are large language models pre-trained on large datasets using ML algorithms. When used with a service like Bedrock, that means you can ensure that all the work you're doing with a model, like Anthropic or Cohere, for example, never leaves the AWS ecosystem. This kind of arrangement is going to be critical for the industries we mentioned because in some cases you're going to need to have the kind of logging and data guarantees that are only possible with an auditable, single infrastructure, like AWS.

You can see how so much of this comes back to the data, where it lives, what you're doing to it, and what sort of privacy guarantees you need to maintain within your Gen AI solution. You may also have noticed that cloud infrastructure, like AWS, is quite well-suited for scaling a solution. If you need to process thousands of documents, fine-tune your LLM, train it against specific examples, or what have you... all of that takes compute power and flexibility. Otherwise you won't be able to iterate on your solution quickly enough to experiment, refine, and get to something that works for your use case.

Outro

Okay, we've covered a lot today. We're going to end the episode here. But I just wanted to end by making a pitch for my team. A lot of what I've talked about can sound daunting, especially if you don't have machine learning and data science experts at hand to develop it for you.

You may know, for instance, your business has a great opportunity for IDP, but you don't have the team for building something beyond an MVP, for example.

If that's you, reach out to us... No really, I mean it. We'll take an hour with you, free of charge, and talk about your objectives, use case, and how you might go about leveraging AWS to that end. We commonly work with customers that have just an idea, a business objective, or an opportunity they're not sure how to capitalize on.

If any of that sounds familiar, you should drop us a line. You can visit us any time on the web at mission cloud dot com.

Best of luck out there and happy building!