985 B
985 B
PDF to Markdown
There are numerous instances where we need to process PDFs into Markdown to enable programmatic extraction and modern pipelines. This repository provides a minimal environment for performing these computations locally (requires access to enterprise GPUs) or via datalab API calls.
The scratch/ directory can be used to run jobs as it is ignored by git.
Datalab
Datalab offers a pay-as-you-go service with $10 of free credits per month. It is $0.01 per page when converting to Markdown with their "Accurate" quality.
Set your API key as an environment variable.
export DATALAB_API_KEY=your_api_key_here
We also prefer setting a consistent Datalab pipeline ID as an environment variable.
export DATALAB_PIPELINE_ID=your_pipeline_id_here
You can then run your API call with:
pixi run -e datalab datalab custom-pipeline --pipeline_id "${DATALAB_PIPELINE_ID}" --format markdown PATH