From cf561dbe00c1680064d33fd49bf1ac31459449e3 Mon Sep 17 00:00:00 2001 From: Alex Maldonado Date: Tue, 30 Jun 2026 11:57:38 -0400 Subject: [PATCH] feat: add basic README --- README.md | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..0139156 --- /dev/null +++ b/README.md @@ -0,0 +1,29 @@ +# PDF to Markdown + +There are numerous instances where we need to process PDFs into Markdown to enable programmatic extraction and modern pipelines. +This repository is a minimal environment to perform these computations locally (requires access to enterprise GPUs) or with [datalab](https://datalab.to) API calls. + +The `scratch/` directory can be used to run jobs as it is ignored by git. + +## Datalab + +[Datalab](https://datalab.to) offers a pay-as-you-go service with $10 of free credits per month. +It is $0.01 per page when converting to Markdown with their "Accurate" quality. + +Set your API key as an environment variable. + +```sh +export DATALAB_API_KEY=your_api_key_here +``` + +We also prefer setting a consistent Datalab pipeline ID as an environment variable. + +```sh +export DATALAB_PIPELINE_ID=your_pipeline_id_here +``` + +You can then run your API call with: + +```sh +pixi run -e datalab datalab custom-pipeline --pipeline_id "${DATALAB_PIPELINE_ID}" --format markdown PATH +```