Dictionary and recipesActive 2026AA releaseLocal controlled test

What cTAKES can find depends on the dictionary it actually loads

The UMLS release, the local cTAKES dictionary, and the WSD model are related but different artifacts. Each has its own bytes, version, hash, and job.

UMLS: 2026AA
SNOMEDCT_US: 2026_03_01
RXNORM: 20AA_260302F
LNC: 282
Dictionary DB: HSQL + indexed CACHED tables
Distribution: Not approved publicly

UMLS release, local dictionary, and WSD model answer different questions

Object	What it contains	What the run uses it for
UMLS 2026AA META files	The source Metathesaurus tables such as MRCONSO, MRSAB, MRSTY, MRDEF, MRRANK, and MRREL.	Input to controlled dictionary and WSD builds. The raw release is not queried for every note.
cTAKES dictionary database	The selected terms, CUIs, semantic types, preferred terms, and RxNorm source-code table in cTAKES HSQL shape.	Fast phrase lookup while cTAKES processes the note.
Graph/context WSD SQLite model	Profiles derived from the exact dictionary candidates plus ranked UMLS terms, definitions, types, and bounded relations.	Compare same-span candidates after lookup has created them.

If a concept is absent from the cTAKES dictionary, WSD cannot bring it back later. WSD only compares the candidates that dictionary lookup already attached to the span.

UMLS source files are filtered into a local cTAKES dictionary; phrase lookup creates CUI and RxCUI candidates; the WSD model compares only candidates already created. — Figure 1. The same dictionary database hash is bound into the run and the WSD model receipt.

The source files used to build the dictionary

Build term	Plain-language meaning
`MRCONSO.RRF`	The UMLS table of concept names and source terms. A row connects wording, a CUI, a source vocabulary, a language, and a term type.
`MRSAB.RRF`	The UMLS source-vocabulary inventory. The build reads it to prove which SNOMED CT, RxNorm, and LOINC releases are current and included.
`SAB`	Source abbreviation on a terminology row, such as `SNOMEDCT_US`, `RXNORM`, or `LNC`.
`TTY`	Term type: the source vocabulary's label for the kind of name, such as preferred term, synonym, ingredient, or official LOINC name.
`HSQL`	HSQLDB, the Java relational-database format used by this cTAKES dictionary.
`MEMORY`	An HSQL table type whose contents are materialized in Java heap when the database opens.
`CACHED`	An HSQL table type that keeps persisted rows on disk and uses a bounded cache plus indexes instead of loading the complete table into heap.

Source selection happens before cTAKES DictionaryBuilder runs

The preparation step filters MRCONSO by language, source, source-specific term type, suppression, and allowed semantic types before cTAKES DictionaryBuilder runs. The builder’s code-table argument is a separate choice, so the workflow passes RXNORM as the source-code table cTAKES should create.

Selection	Meaning	Reason
English rows	MRCONSO language is English.	The source notes and downstream models are English.
Selected SAB	SNOMEDCT_US, RXNORM, and LNC.	Clinical concepts, medication identities, and LOINC concepts requested for this release.
Source-specific TTY	Term types are chosen separately for each source rather than applying one generic list blindly.	Each vocabulary uses its own term-type conventions.
Suppression filter	Suppressed terms are excluded according to the release profile.	Prevents known-retired or suppressed strings from entering routine lookup.
Semantic-type filter	Allowed TUIs constrain the clinical domain included in the dictionary.	Reduces unsupported or irrelevant candidate classes before runtime.
Code target `RXNORM`	Only the RxNorm source-code table is requested.	The delivery needs CUI/RxCUI. Unused SNOMED and LOINC source-code tables would add size without a consuming output field.

Exact 2026AA source profile used for this candidate release

UMLS source (`SAB`)	Included term types (`TTY`)	What those labels mean
`SNOMEDCT_US`	`PT,SY`	Designated preferred names and designated synonyms.
`RXNORM`	`BN,IN,MIN,PIN`	Brand name, ingredient, multi-ingredient name, and precise ingredient name.
`LNC`	`LC,LN,OSN`	Long common name, LOINC official fully specified name, and official short name.
Suppression	`SUPPRESS=N`	Only rows marked not suppressed are retained.
Code-table target	`RXNORM`	The builder persists the RxNorm source-code lookup needed for RxCUI output; this is not another term-type list.

Allowed semantic types (TUI). T017, T018, T019, T020, T021, T022, T023, T024, T025, T026, T029, T030, T031, T033, T034, T037, T046, T047, T048, T049, T059, T060, T061, T074, T079, T080, T081, T082, T109, T110, T111, T114, T115, T116, T118, T119, T120, T121, T122, T123, T124, T125, T126, T127, T130, T184, T190, T191.

This is the literal reviewed profile, including configured TUIs that happened to have no selected rows in this release. The source inventory records that distinction instead of silently rewriting the profile after the build.

Prove the source release, not only the folder name

The builder reads MRSAB and accepts one current row per selected source where both CURVER=Y and SABIN=Y. A missing row or more than one accepted current row fails the build. That prevents a historical source row from being recorded as current.

Read the mechanism: scripts/prepare_dictionary_source.py applies the source filters, and the content-addressed release record is dictionaries/usd-star-2026AA-r1/manifest.json.

The build creates a candidate release without changing the active runtime

bash scripts/build_dictionary_full.sh \
  --umls /path/to/umls-2026AA/2026AA/META \
  --ctakes-home /approved/apache-ctakes-6.0.0

Validate the required META files and record their hashes without copying licensed terminology into the receipt.
Prepare a filtered dictionary source tree using the reviewed source, term-type, suppression, and semantic-type rules.
Run cTAKES DictionaryBuilder in an isolated candidate directory.
Open the completed HSQL database and audit table presence, counts, identities, and indexes.
Convert startup-loaded MEMORY tables to indexed CACHED storage.
Export a deterministic WSD seed from that exact validated database.
Write build, validation, inventory, storage-optimization, and strict manifest receipts.
Write activate_command.txt only when every required phase passes.

Read the mechanism: scripts/build_dictionary_full.sh, tools/dictionary/DictionaryDatabaseAudit.java, and tools/dictionary/DictionaryStorageOptimizer.java.

The accepted database has measured contents and storage behavior

CUI_TERMS rows: 469,083
TUI rows: 319,935
PREFTERM rows: 271,611
RXNORM rows: 26,015
Database SHA-256: 5a5e7c8d…e581
Rare lookup p50 / p95: 54 µs / 1,577 µs

The lookup values are database microbenchmarks, not end-to-end note speed. They measure selected queries against the built database. JVM startup, model initialization, note length, annotation count, writer profile, and host load remain separate.

Why CACHED tables matter

Large HSQL MEMORY tables are materialized into heap when the database opens. Indexed CACHED tables keep persisted data on disk with cached access and explicit indexes. The optimizer converts only the tables recorded as safe for this dictionary layout, reopens the database, reruns the audit, and records the resulting database hash.

The WSD seed is exported after this validation from the exact database that will be activated. That closes the gap where a WSD model could be built from an earlier dictionary script while cTAKES later loaded different bytes.

Activation is a separate, atomic operation

Activation requires cTAKES to be stopped. The script verifies the candidate manifest and database again, stages the new descriptor/database, records the prior active selection for rollback, and moves the active pointer only after the candidate is in place.

Activation receipt	Why it is recorded
Candidate manifest hash	Proves which reviewed candidate was selected.
Dictionary database hash	Proves the runtime bytes match the candidate manifest.
Descriptor paths/hashes	Proves which XML points cTAKES to those bytes.
Prior active release	Provides a concrete rollback target.
Activation time and status	Separates build completion from the moment the runtime selection changed.

The current literal dictionary release is usd-star-2026AA-r1. It remains in the hashed, content-addressed manifest and composite release ID. The manifest is not digitally signed. USDHUB is the workflow name; renaming this dictionary artifact requires a new manifest, activation receipt, WSD binding, analysis identity, and acceptance run.

Read the activation mechanism: scripts/activate_dictionary.sh and dictionaries/active_release.json.

The recipe decides which engines consume that dictionary output

A dictionary and recipe are not interchangeable. The dictionary says which phrase-to-concept candidates exist. The recipe says when lookup runs, which later models add attributes or relations, and which writers serialize the result.

The recipe orders source structure, dictionary lookup, WSD, Drug NER, assertions, relations, and writers. — Figure 2. The reviewed USDHUB stage set is stored at `pipelines/production/star.piper`; the internal key remains `star`.

Recipe choice	Current default
Text structure	Stock section, sentence, token, POS, and chunk components; fallback segment only when no section exists.
Lookup	Selected 2026AA dictionary release.
WSD	Candidate-preserving graph/context scorer with validated active model; `--no-wsd` remains the explicit comparison override.
Medication	Stock Drug NER through a synchronized wrapper, before assertion models.
Assertions	Only the five requested stock ClearTK models: polarity, uncertainty, conditional, generic, subject.
Relations	Stock degree and location models; self-links removed at serialization.
CAS writers	Canonical SQLite by default; native XMI is added when `--writers standard` is selected.
Post-run export	The checked concept and drug CSV files are streamed from completed SQLite views.

Return to the clinician-level stage explanation, or follow the dictionary candidates into the default WSD step. The recipe source is pipelines/production/star.piper.

The manifest proves bytes; it does not grant redistribution rights

The local dictionary and WSD model contain or derive from licensed terminology. The manifest records the selected sources, versions, file/database hashes, filters, counts, build phases, and required notice decisions. It cannot replace the accountable license and distribution review.