How it is built

The structure, and the rules it enforces

Twenty-three tables, 485 controlled terms across 36 vocabularies, and a short list of things the database will not let you do.

The shape follows the evidence rather than the convenience of a spreadsheet. A person has casualty events; an event has injuries; an injury has treatments, and possibly an amputation, and its own outcome. Held deliberately to one side is the newspaper itself — issue, page, article, mention — so that every assertion in the first chain is reachable from a particular column of newsprint in the second.

personcasualty eventinjurytreatmentamputation

issuepagearticlesource mentioninjury mention

The tables

The people and what happened to them

Table	Columns
`persons`	17	One row for a person, however little of the name survives.
`name_as_printed`	8	The name exactly as set in type, kept apart from any expansion.
`military_service`	27	Allegiance, arm, unit, vessel, battery. The paper's own word kept beside the normalised one.
`casualty_events`	38	What befell the person, when, where, and by what general cause.
`injuries`	54	One row per distinct wound. Three wounds are three rows.
`injury_treatments`	14	Dressing, hospital, surgery, tied to the wound treated.
`amputations`	16	Tied to the injury that caused it.
`projectiles`	11	Ball, shell, fragment, splinter, in period terms.

The evidence

Table	Columns	Rows
`publications`	7	2
`issues`	18	2,192	Every date in the study, whether or not a copy has been found.
`pages`	14	18	Page image, OCR, and the measured legibility of each.
`articles`	12	0	Headline, column, dateline, and the voice the report speaks in.
`source_mentions`	18	0	The passage of newsprint an assertion came from, with untouched OCR.
`injury_mentions`	10	0	Three reports of one arm wound are three mentions of one injury.
`places`	12	0	Location as printed beside the standardised place.
`external_verifications`	13	0	Later research, held apart from what the paper said.

Housekeeping

Table	Columns	Rows
`review_queue`	12	4	Anything uncertain, waiting for a person to decide.
`audit_log`	11	0	Every change, with its reason and its author.
`candidates`	10	70	Passages worth reading. Not records.
`processing_checkpoints`	7	28	Where a run of processing got to.
`vocab_domain`	2	36
`vocab_term`	6	485	The controlled vocabularies.
`incidents`	5	0

The rules

Nothing is invented

No name, rank, company, regiment, battery, vessel, date, place, weapon, calibre, wound or hospital is supplied because it seems probable. Where a thing cannot be established it is recorded as unknown, and unknown is a perfectly good answer.

The paper is never silently corrected

What the Mercury reported and what later evidence showed are separate columns. A captain reported killed who walks home keeps both statements, in the order they were made.

Original text is never overwritten

The repository’s OCR, a local OCR pass, and a corrected transcription are three fields. Correcting a reading never destroys the reading it replaced.

Wounds are counted one at a time

“Wounded in the arm and leg” is two injuries. “Wounded severely in the leg” is one. Where a single ball passed through a hand and into a thigh, the two wounds are linked so that they are never counted as two shots.

Period wording is kept

“A piece of shell” is stored as it was printed and normalised to shell fragment beside it. A rifle projectile is not called a Minié ball because it was fired from a rifle; only because the paper said so.

Nothing is merged on a surname

Two men called Smith are two men until rank, company, unit, date or engagement says otherwise. Ambiguous cases go to a person, not to a confidence score.

Absence of leave is not a casualty

Deserted, absent, straggling, furloughed, detached and transferred are excluded by the discovery pass, and flagged where they appear beside casualty language.

Every change leaves a trace

The audit log records the old value, the new value, who changed it, why, and on what authority.

The pipeline

01Locate an issue and record it, found or not
02Fetch the scan and checksum it
03OCR it, keeping the repository's own text untouched
04Judge the OCR, and stop here if it cannot be read
05Segment the columns and the articles
06Find the passages carrying casualty language
07Extract people, then events, then injuries one by one
08Extract weapon, projectile and immediate mechanism
09Extract treatment, surgery, amputation and outcome
10Resolve repeated mentions without unsafe merging
11Put everything uncertain in front of a person
12Check the finished record against the page image

Controlled vocabularies

These are tables rather than fixed constraints, so a term met in the paper and not anticipated can be added with a note on where it came from, instead of being quietly forced into the nearest existing box.

Vocabulary	Terms
`allegiance`	8	Which side, established from context and not from the paper's tone.
`amputation_level`	10	Level at which a limb was taken off.
`anatomical_region`	34	Principal region of the body.
`article_type`	21	Kind of newspaper item the mention sits in.
`body_side`	5	Side of the body.
`branch`	23	Arm of service. The newspaper's own word is kept separately.
`casualty_status`	40	What was reported to have befallen the person.
`cause_category`	13	Top of the causation hierarchy.
`civilian_or_military`	8	Broad class of person.
`completeness_status`	7	How complete a located issue is.
`date_precision`	7	How firmly the date of the event is fixed.
`direct_or_indirect`	6	Whether the weapon acted on the body directly.
`external_source_type`	13	Class of record used for later verification.
`fire_source`	6	Whose fire, where the source is actually stated.
`injury_mechanism`	73	The thing that did the damage to the body.
`injury_outcome`	13	Outcome of one injury, not of the person.
`name_expansion_status`	4	Whether an abbreviated name has been expanded.
`ocr_availability`	5	Whether machine-readable text exists.
`ocr_quality`	5	Measured usefulness of the OCR for extraction.
`processing_stage`	13	Stage of the ingestion pipeline.
`projectile_family`	10	Coarse grouping of projectiles for counting.
`projectile_type`	27	The projectile, in period terms where possible.
`publication_status`	5	Whether the paper was publishing on a date.
`review_action`	7	What a reviewer did.
`review_reason`	19	Why a row was put in the review queue.
`review_status`	4	Where a review stands.
`reviewer_status`	6	State of human review of a mention.
`scan_availability`	5	Whether a page image exists.
`severity`	7	Reported severity of a wound.
`source_perspective`	9	Voice the report speaks in.
`treatment_type`	12	Kind of medical attention recorded.
`verification_status`	6	State of checking against evidence.
`weapon_system`	23	The weapon, where one is actually named.
`wound_type`	24	Character of the wound as described.
`yes_no_possibly_unknown`	4	Four-valued flag.
`yes_no_unknown`	3	Three-valued flag.