PDF to Markdown for RAG: a worked example

Most RAG tutorials skip the boring step: getting clean text out of the PDF. If the text comes out with headings flattened, tables turned into a soup of numbers and chunks cut in the middle of a sentence, no embedding model will save the answers.

Here is that step done on a real document, the arXiv paper "Attention Is All You Need" (1706.03762). It is 15 pages, two columns, with formulas, three tables and a long reference list. Everything below is real output from my PDF to Markdown actor on Apify, not a cleaned-up example.

The input

One URL and chunking turned on:

{
  "pdfs": ["https://arxiv.org/pdf/1706.03762v7"],
  "outputChunks": true,
  "chunkSize": 2000,
  "chunkOverlap": 200
}

The paper has a real text layer, so no OCR runs. The result is one dataset item with the whole document as markdown, plus chunks and chunksMeta.

The Markdown

The run gave 40,262 characters of Markdown for 15 pages. The heading tree comes back from the layout, numbered sections included:

## Attention Is All You Need
### Abstract
### 1 Introduction
### 2 Background
### 3 Model Architecture
#### 3.1 Encoder and Decoder Stacks
#### 3.2 Attention
##### 3.2.1 Scaled Dot-Product Attention
##### 3.2.2 Multi-Head Attention

That matters for RAG more than it looks. A heading is the cheapest context you can give a chunk, and in a two-column paper the section titles are just bold text at body size, so a plain text extractor loses them.

Table 1 of the paper comes out as a real Markdown table (the paper's multiplication dot is shown as x here):

| Layer Type | Complexity per Layer | Sequential Operations | Maximum Path Length |
| --- | --- | --- | --- |
| Self-Attention | O(n2 x d) | O(1) | O(1) |
| Recurrent | O(n x d2) | O(n) | O(n) |
| Convolutional | O(k x n x d2) | O(1) | O(logk(n)) |
| Self-Attention (restricted) | O(r x n x d) | O(1) | O(n/r) |

Two honest notes on this. Superscripts are flattened, so n squared reads as n2. And Table 2, which has a two-level header (BLEU and Training Cost, each split into EN-DE and EN-FR), comes out with the second header level as its own row. An LLM reads both fine, but if your questions depend on exact formulas, keep the original PDF next to the text.

The chunks

With chunkSize 2000 the paper becomes 23 chunks between 1,315 and 2,147 characters. Each chunk ends at the best boundary in range: before a heading first, then a paragraph break, then a sentence end. No chunk ends on a dangling heading, and no table is ever split across two chunks.

Every chunk has a metadata entry in the same order:

{ "index": 6, "pageStart": 4, "pageEnd": 5, "headings": ["Attention Is All You Need", "3 Model Architecture", "3.2 Attention", "3.2.1 Scaled Dot-Product Attention"] }

That is the part I built this for. When a retriever returns chunk 6, the answer can say "page 4 to 5, section 3.2.1 Scaled Dot-Product Attention" instead of "somewhere in the paper".

Loading it into a vector store

The shape maps directly onto any document loader. In Python with LangChain it is a few lines:

from apify_client import ApifyClient
from langchain_core.documents import Document

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("pankovs/pdf-to-llm-markdown").call(run_input={
    "pdfs": ["https://arxiv.org/pdf/1706.03762v7"],
    "outputChunks": True,
})

docs = []
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    for text, meta in zip(item["chunks"], item["chunksMeta"]):
        docs.append(Document(page_content=text, metadata={
            "source": item["url"],
            "pages": f'{meta["pageStart"]}-{meta["pageEnd"]}',
            "section": " > ".join(meta["headings"]),
        }))

From there docs goes into whatever store you use. It is also worth putting the section path in front of the chunk text before embedding, so a short chunk under a descriptive heading still carries that heading's meaning.

What it cost

Pricing is per event: $0.00416 per document and $0.000192 per text page, plus Apify's start event of $0.00005. This paper is 15 text pages, so the run cost about $0.007. A scanned page goes through OCR at $0.01, and pages where OCR finds nothing usable are not charged.

When it is not the right tool

Formulas stay plain text, tables on scanned pages stay paragraphs, and multi-level table headers become separate rows. If you need LaTeX out of a math paper, a model-based converter will do better. If you need a lot of normal business PDFs (contracts, manuals, reports, filings) turned into chunks you can cite, this is the job it does.

The actor on Apify has the full input reference. Its page on this site shows more real samples, including a scanned page, a Russian document and an Arabic one, and every release. Questions or a PDF that comes out wrong: info@pankovs.com.