First implementation (#1)
Reviewed-on: #1
This commit was merged in pull request #1.
This commit is contained in:
1 parent
34c0abb26a
commit
4fa326dc76
534 files changed
+28280
-37
No files matched your search
@@ -0,0 +1,127 @@
|
||||
# 3. Nodes
|
||||
|
||||
A document is a tree of nodes.
|
||||
`nodes.jsonl` holds one node per line, in document order: the root first, then each node before its children, and siblings in order.
|
||||
|
||||
## 3.1 The node record
|
||||
|
||||
| Property | Required | Holds |
|
||||
| --- | --- | --- |
|
||||
| `id` | Yes | The node ID ([4](04-identity.md)). |
|
||||
| `kind` | Yes | One of the kinds in [3.2](#32-kinds-and-children). |
|
||||
| `parent` | Yes | The parent's ID, or null for the root. |
|
||||
| `ord` | Yes | The position among its siblings, from 0. |
|
||||
| `label` | No | The printed label (`2.3`, `Figure 2`, `(1)`, `a`), for kinds that take one. |
|
||||
| `attrs` | No | Structural attributes, by kind ([3.4](#34-attributes)). |
|
||||
| `content` | No | The node's own content, by kind ([3.3](#33-content)). |
|
||||
| `withheld` | No | `true` when the policy withholds the content ([12.5](12-package.md#125-text-policies)). |
|
||||
| `fidelity` | No | For leaves, the level the pipeline established: `F0`, `F1` or `F2` ([8.1](08-fidelity.md#81-levels-of-fidelity)). |
|
||||
| `prov` | No | `{witnesses, backend?, reading?, corroborated_by?}`: the witness IDs the text came from, the backend, and the witnesses that agree. |
|
||||
| `text_hash`, `match_hash`, `align_key`, `own_hash`, `node_hash` | Yes | The five content values ([5](05-hashes.md)). |
|
||||
|
||||
Optional properties are omitted, never written as null.
|
||||
The schema is `node.schema.json`.
|
||||
|
||||
## 3.2 Kinds and children
|
||||
|
||||
| Kind | May contain | Content | Label |
|
||||
| --- | --- | --- | --- |
|
||||
| `document` | `front`, `section`, `back` | | |
|
||||
| `front` | `title`, `authors`, `abstract`, `keywords`, `notes` | | |
|
||||
| `title` | `sentence` | | |
|
||||
| `authors` | | runs | |
|
||||
| `abstract` | blocks, `section` | | |
|
||||
| `keywords` | | runs | |
|
||||
| `notes` | `footnote`, `paragraph` | | |
|
||||
| `section` | `heading` first, then blocks and `section` | | Yes |
|
||||
| `heading` | `sentence` | | |
|
||||
| `paragraph` | `sentence` | | |
|
||||
| `sentence` | `equation` | runs | |
|
||||
| `equation` | `equation-line` | math | Yes |
|
||||
| `equation-line` | | math | Yes |
|
||||
| `figure` | `panel`, `caption`, `figure-note` | | Yes |
|
||||
| `panel` | | | Yes |
|
||||
| `caption` | `sentence` | | |
|
||||
| `figure-note` | `sentence` | | |
|
||||
| `table` | `caption`, `table-head`, `table-body`, `table-note` | | Yes |
|
||||
| `table-head`, `table-body` | `row` | | |
|
||||
| `table-note` | `sentence` | | |
|
||||
| `row` | `cell` | | |
|
||||
| `cell` | | runs | |
|
||||
| `list` | `item` | | |
|
||||
| `item`, `quote` | blocks | | |
|
||||
| `code`, `algorithm` | | verbatim | Yes |
|
||||
| `theorem` | blocks | | Yes |
|
||||
| `footnote` | `sentence` | | Yes |
|
||||
| `back` | `acknowledgments`, `references`, `appendix`, `notes` | | |
|
||||
| `acknowledgments` | blocks | | |
|
||||
| `references` | `reference` | | |
|
||||
| `reference` | | runs | Yes |
|
||||
| `appendix` | `section` | | |
|
||||
|
||||
Blocks are `paragraph`, `equation`, `figure`, `table`, `list`, `quote`, `code`, `algorithm`, `theorem` and `footnote`.
|
||||
|
||||
These rules also hold:
|
||||
|
||||
1. Exactly one node is a root, and it is a `document`.
|
||||
2. A `heading` is a section's first child, and a section has at most one.
|
||||
3. `front`, `back`, `title`, `authors`, `abstract`, `keywords`, `caption`, `table-head`, `table-body`, `acknowledgments` and `references` occur at most once among their siblings.
|
||||
4. A `table` has exactly one `table-body`.
|
||||
5. `title`, `heading`, `paragraph`, `caption`, `figure-note`, `table-note`, `table-head`, `table-body`, `row`, `list`, `item`, `quote`, `theorem`, `footnote` and `appendix` have at least one child.
|
||||
6. Only the kinds marked in the table take a `label`.
|
||||
7. A body without headings, as in a letter, is a section with no heading.
|
||||
|
||||
Kinds outside this list are refused; a new kind is a minor version.
|
||||
Page breaks, running heads and columns are not nodes: they are geometry ([7](07-geometry.md)).
|
||||
|
||||
## 3.3 Content
|
||||
|
||||
| Content | Kinds | Shape |
|
||||
| --- | --- | --- |
|
||||
| Runs | `sentence`, `cell`, `reference`, `authors`, `keywords` | A non-empty list of inline runs ([3.5](#35-inline-runs)). |
|
||||
| Math | `equation`, `equation-line` | `{notation, source}`: `tex` or `typst`, and the source as written. |
|
||||
| Verbatim | `code`, `algorithm` | `{language?, text}`. |
|
||||
|
||||
Other kinds have no `content`.
|
||||
Derived renderings, such as MathML, are not content; they live in `derived/` ([12.7](12-package.md#127-derived-files)).
|
||||
|
||||
## 3.4 Attributes
|
||||
|
||||
| Kind | Attribute | Holds |
|
||||
| --- | --- | --- |
|
||||
| `section` | `level` (required) | 1 for a section, 2 for a subsection, and so on. |
|
||||
| `table` | `columns` | The column definitions: `[{id, header_path}]`, where `header_path` lists header cell IDs, outermost first ([9.2](09-figures-tables-equations.md#92-tables)). |
|
||||
| `cell` | `column`, `rowspan`, `colspan`, `role` | The column's ID, spans (default 1), and `data` (default), `column-header`, `row-header` or `row-group-header`. |
|
||||
| `row` | `header_path` | The cell IDs that label the row, outermost first. |
|
||||
| `list` | `ordered` | Whether the list is numbered. |
|
||||
| `item` | `marker` | The printed marker. |
|
||||
| `theorem` | `variant` | The printed kind: Theorem, Lemma, Definition, Proof, Remark. |
|
||||
| `panel` | `image`, `crop`, `figure_text` | The image asset, a crop, and the text found inside the panel ([9.1](09-figures-tables-equations.md#91-figures-and-panels)). |
|
||||
|
||||
No other kind carries `attrs`, and each kind carries only its own attributes.
|
||||
|
||||
## 3.5 Inline runs
|
||||
|
||||
| Run | Properties | Plain projection |
|
||||
| --- | --- | --- |
|
||||
| `text` | `text`, and optional `style`: a sorted list of distinct values from `em`, `strong`, `sup`, `sub`, `sc`, `mono`, never both `sup` and `sub`. | The text. |
|
||||
| `math` | `notation` (`tex` or `typst`), `source`. | `$` + source + `$` |
|
||||
| `cite` | `refs`: reference node IDs; `text`: the printed form, such as `23`, `[1,2]` or `(Smith, 2019)`. | The printed form. |
|
||||
| `xref` | `target`: a node ID in this document, an address in another, or null when not yet resolved; `text`. | The printed form. |
|
||||
| `fnmark` | `footnote`: a footnote's ID, or null; `text`: the mark. | The mark. |
|
||||
| `link` | `url`, `text`. | The text. |
|
||||
| `equation` | `node`: the ID of an equation child of the sentence. | `$$` + that equation's source + `$$` |
|
||||
|
||||
Runs are canonical: no `text` run is empty, and no two adjacent `text` runs have the same style list.
|
||||
A sentence's `equation` runs name its `equation` children, each once, in the children's order, and a sentence has no other children.
|
||||
A display equation the sentence runs through is thus part of the sentence, while keeping its own ID and hashes.
|
||||
|
||||
## 3.6 The plain projection
|
||||
|
||||
The plain projection of a node with runs is the concatenation of its runs' projections.
|
||||
That of an equation or an equation line is `$$` + source + `$$`, and that of a verbatim node is its text.
|
||||
That of any other node is the plain projections of its text leaves joined with one U+0020.
|
||||
|
||||
A node's text leaves are found by walking its subtree in document order and stopping at `sentence`, `cell`, `reference`, `authors`, `keywords`, `code` and `algorithm` nodes, and at `equation` nodes whose parent is not a sentence; an `equation-line` is never a text leaf.
|
||||
|
||||
Offsets, quotations and the text hash are all defined on the plain projection as stored, before normalization.
|
||||
Reference in new issue
Block a user