30 lines
985 B
Markdown
30 lines
985 B
Markdown
# PDF to Markdown
|
|
|
|
There are numerous instances where we need to process PDFs into Markdown to enable programmatic extraction and modern pipelines.
|
|
This repository provides a minimal environment for performing these computations locally (requires access to enterprise GPUs) or via [datalab](https://datalab.to) API calls.
|
|
|
|
The `scratch/` directory can be used to run jobs as it is ignored by git.
|
|
|
|
## Datalab
|
|
|
|
[Datalab](https://datalab.to) offers a pay-as-you-go service with $10 of free credits per month.
|
|
It is $0.01 per page when converting to Markdown with their "Accurate" quality.
|
|
|
|
Set your API key as an environment variable.
|
|
|
|
```sh
|
|
export DATALAB_API_KEY=your_api_key_here
|
|
```
|
|
|
|
We also prefer setting a consistent Datalab pipeline ID as an environment variable.
|
|
|
|
```sh
|
|
export DATALAB_PIPELINE_ID=your_pipeline_id_here
|
|
```
|
|
|
|
You can then run your API call with:
|
|
|
|
```sh
|
|
pixi run -e datalab datalab custom-pipeline --pipeline_id "${DATALAB_PIPELINE_ID}" --format markdown PATH
|
|
```
|