Boilerplate Buries the Content
You’re building a content aggregator, a newsletter curation tool, or an article archival system. You have documents — PDFs of articles, Word exports, saved web pages. You need the clean text: title, author, publication date, and the article body. Nothing else.
What you get instead is headers, footers, navigation menus, sidebars, copyright notices, ad placeholders, and page numbers mixed in with the actual content. A raw text extraction from a PDF gives you everything on the page, in reading order, with no indication of what’s content and what’s chrome.
The Document Extraction API extracts semantic fields — not raw text. Define a schema for article content, and the parser pulls out the title, author, date, and body text, ignoring the noise around it.
Article Extraction Schema
import { IterationLayer } from "iterationlayer";
const client = new IterationLayer({ apiKey: "YOUR_API_KEY" });
const result = await client.extract({
files: [
{ type: "url", name: "article.pdf", url: "https://example.com/article.pdf" }
],
schema: {
fields: [
{
name: "title",
type: "TEXT",
description: "Article title or headline",
is_required: true,
},
{
name: "author",
type: "TEXT",
description: "Author name",
},
{
name: "publish_date",
type: "DATE",
description: "Publication date of the article",
},
{
name: "body",
type: "TEXTAREA",
description: "Main article text content, excluding headers, footers, sidebars, and navigation",
is_required: true,
},
{
name: "summary",
type: "TEXT",
description: "Brief summary or abstract of the article if present",
max_length: 500,
},
{
name: "category",
type: "TEXT",
description: "Article category or section (e.g., Technology, Business, Science)",
},
],
},
});The TEXTAREA field type handles long-form text — the article body can be thousands of words. The parser separates content from boilerplate based on the field description, not layout rules.
Clean Output
{
"title": {
"type": "TEXT",
"value": "The Quiet Revolution in Battery Chemistry",
"confidence": 0.97
},
"author": {
"type": "TEXT",
"value": "James Park",
"confidence": 0.94
},
"publish_date": {
"type": "DATE",
"value": "2026-01-15",
"confidence": 0.93
},
"body": {
"type": "TEXTAREA",
"value": "Solid-state batteries have been five years away for the last twenty years. But the latest generation of prototypes from three independent labs suggests the timeline might finally be real...",
"confidence": 0.91
},
"summary": {
"type": "TEXTAREA",
"value": "Recent advances in solid-state battery technology suggest commercial viability within three years, driven by breakthroughs in solid electrolyte materials.",
"confidence": 0.88
}
}The body text is clean — no page numbers, no headers, no footer disclaimers. The confidence score on the body is typically slightly lower than on structured fields like title and date, because extracting the “main content” requires judgment about what to include and exclude.
Multiple Document Formats
Content comes in many formats:
- PDF articles — academic papers, magazine exports, report PDFs
- DOCX files — Word documents, manuscript drafts
- Text files — markdown, plain text exports
- Scanned articles — OCR handles photos or scans of printed articles
The same schema works for all of them. A PDF with two-column layout and a single-column Word document both get extracted into the same clean structure.
What Gets Excluded
The value of clean extraction isn’t just what you get — it’s what you don’t get. Understanding what the parser filters out helps you trust the output.
Headers and footers. Page numbers, running titles, journal names repeated on every page, copyright lines. These appear on every page of a PDF but aren’t part of the article content. Raw text extraction includes them. The parser doesn’t.
Sidebars and pull quotes. Magazine-style PDFs often have sidebars with related content, pull quotes duplicating text from the main body, or author bios set off in a colored box. These are visually distinct from the main flow. The parser recognizes them as separate from the body content.
Navigation and metadata blocks. Saved web pages include navigation menus, breadcrumbs, related article links, and social sharing buttons. These are chrome, not content. The parser extracts the article body and ignores the page scaffolding.
Footnotes and endnotes. These are a judgment call. Academic papers have footnotes that are essential to the content. Magazine articles have footnotes that are tangential. The parser includes footnotes that are inline with the text flow and excludes page-level footer content. If you need explicit control, add a separate TEXTAREA field for “footnotes and endnotes” and the parser will extract them independently.
Advertisements. PDF exports from news sites sometimes include ad placeholders or sponsored content blocks mixed in with editorial content. The parser distinguishes editorial content from promotional blocks based on the field description’s focus on “main article text.”
Handling Different Content Types
Not all articles are the same. The schema adapts to different content types with minor adjustments.
For academic papers, add fields for abstract, keywords, and references:
const academicSchema = {
fields: [
{ name: "title", type: "TEXT", description: "Paper title", is_required: true },
{ name: "authors", type: "ARRAY", description: "Paper authors", item_schema: {
fields: [
{ name: "author_name", type: "TEXT", description: "Author name" },
{ name: "affiliation", type: "TEXT", description: "Author's institutional affiliation" },
],
}},
{ name: "abstract", type: "TEXTAREA", description: "Paper abstract" },
{ name: "body", type: "TEXTAREA", description: "Main paper text excluding abstract, references, and appendices", is_required: true },
{ name: "keywords", type: "ARRAY", description: "Paper keywords", item_schema: {
fields: [
{ name: "keyword", type: "TEXT", description: "Keyword or key phrase" },
],
}},
{ name: "publish_date", type: "DATE", description: "Publication date" },
{ name: "journal", type: "TEXT", description: "Journal or conference name" },
],
};For news articles, add a source and byline:
const newsSchema = {
fields: [
{ name: "headline", type: "TEXT", description: "Article headline", is_required: true },
{ name: "byline", type: "TEXT", description: "Author name and title" },
{ name: "publish_date", type: "DATE", description: "Publication date" },
{ name: "source", type: "TEXT", description: "Publication or news outlet name" },
{ name: "body", type: "TEXTAREA", description: "Article body text", is_required: true },
{ name: "dateline", type: "TEXT", description: "Dateline location if present (e.g., LONDON)" },
],
};The core pattern is the same: define what you want, let the parser find it, ignore the noise.
Cleaning Strategies for Downstream Use
The extracted text is clean, but you might need further processing depending on your use case.
Search indexing. The body text goes directly into your search index. Strip any remaining whitespace normalization issues (double spaces, trailing newlines) and index the result. The title, author, and date become structured metadata fields in your index.
Summarization. Feed the extracted body to a summarization model. Because the text is already clean — no headers, no footers, no navigation — the model works on pure content. Summarizing raw extracted text often produces summaries that reference page numbers or include fragments of navigation menus.
Deduplication. When aggregating from multiple sources, the same article might appear in different formats — a PDF from one source, a DOCX from another. Clean extraction normalizes the content so you can compare body text across formats for deduplication.
Content Aggregation Pipeline
Batch processing makes this practical for aggregation workflows. Collect 20 articles, send them in a single batch request, and get structured results for all of them:
const articles = [
{ type: "url", name: "article-1.pdf", url: "https://source.example.com/article-1.pdf" },
{ type: "url", name: "article-2.pdf", url: "https://source.example.com/article-2.pdf" },
// ... up to 20 files per batch
];
const result = await client.extract({ files: articles, schema });Use Cases
Newsletter curation. Extract titles, summaries, and authors from a batch of articles. Use the structured data to build newsletter layouts automatically.
Content archival. Extract clean text from document archives for full-text search indexing. The structured output maps directly to your search index fields.
Research aggregation. Parse academic papers or reports to extract titles, authors, dates, and abstracts. Build a searchable database of research content.
What’s Next
Extracted article text feeds directly into Document Generation for automated newsletter layouts — same auth, same credit pool.
Get Started
Check the docs for the TEXTAREA field reference and multi-file extraction documentation. The TypeScript and Python SDKs are available for server-side integration.
Sign up for a free account — no credit card required. Try extracting article content from a few of your documents and compare the clean output to raw text extraction.