Multimodal AI: documents, images and sound · Module 1: Multimodal AI: documents, images and sound in practice
OCR and reading order
Learning outcome: Check extracted text against the layout it came from.
Optical character recognition turns visible text into machine-readable text. A document-understanding system may also infer layout and relationships. Both can fail: a character may be wrong, or correct words may be joined in the wrong order.
Inspect columns, footnotes and tables before relying on a summary. Preserve page references so an extracted claim can be traced back. Use a confidence flag or manual review route for unclear scans. Do not let a clean-looking text output erase uncertainty in the image. If a page is cropped, record what is absent rather than assuming the missing edge contains nothing important.
Worked example
A fictional two-column notice has “No booking required” under a drop-in event and “Booking required” under a separate workshop. OCR interleaves the columns and produces one contradictory paragraph. The repair is to restore the section boundaries from the page, then check each event separately. Correct spelling alone would not repair the relationship.
Try it yourself
Create a two-column fictional notice on paper. Transcribe it in reading order and mark the headings that keep the events separate. Add one unclear character and label it unresolved.
You can do this exercise on paper. If using an assistant, keep to the fictional material and an approved tool.
Compare your approach
The transcript should preserve which condition belongs to which event. Keep the unclear character visible as uncertainty. A reviewer should be able to trace every booking requirement back to a specific place on the original.
Check your understanding
Every word was recognised correctly. Can the extracted meaning still be wrong?
- Yes, because layout and reading order may be wrong
- No, spelling accuracy guarantees meaning
- Only if the page contains a photograph
Show the answer and explanation
Answer: Yes, because layout and reading order may be wrong. Correct words can be joined in the wrong order or assigned to the wrong heading. Layout relationships must be checked as well as character recognition.
Workbook
Download the course workbook to record your answer and compare it after the knowledge check.