How one clinical note moves through cTAKES
A clinician writes a sentence. cTAKES marks the clinical phrases in that sentence and returns candidate mappings from the selected terminology. The USDHUB wrapper keeps the note, the extracted rows, the dictionary, and the exact analysis tied together so another person, or another site, can tell how each row was made.
The note we will follow
The same hand-authored note runs through the entire article. It contains a symptom, a symptom the patient does not have, and a medication plan. The codes are fake, so the example can show the mechanism without carrying patient data or licensed terminology content. The note, rows, codes, and WSD scores below are hand-authored fixtures that match the output schema. They are not cTAKES accuracy or WSD evaluation results.
The file begins as ordinary text. There is no diagnosis column, medication table, or standardized code inside it. Those structures appear only after the software finds a phrase, records exactly where it occurred, and asks a terminology dictionary which clinical meanings could match it.
Synthetic note
USDHUB00000000000000000001.SYNTHETIC-DEMO.txt
SYNTHETIC NOTE - no patient data.
Assessment
started yesterday. No .
Plan
Start 10 mg by mouth twice daily for 5 days.
Imaging context
A near the right kidney is described.
Select a highlighted source span. The matching row later in the article selects the same exact characters.
SYNTHETIC-DEMO.txt
cTAKES marks its phrases as annotations
cTAKES creates annotations over the note
Apache cTAKES is clinical text-processing software. It does not replace the note and it does not rewrite the clinician’s sentence. It adds annotations over the original characters. An annotation can say that characters 46 through 62 contain a symptom phrase, that the phrase has one or more terminology candidates, and that another component found a location connected to it.
cTAKES stores the note and all of those annotations in a Common Analysis Structure, usually shortened to CAS. The CAS is the working copy inside the Java process. Each component reads what earlier components added and writes its own annotations back. The final writers then turn that in-memory structure into files that can be reviewed or analyzed.
The source span
For “Right flank pain,” the row keeps the phrase text and its exact begin-and-end character positions in the note. Those two positions are the span, and they let a reviewer return from the row to the words the clinician actually wrote.
The annotation
The annotation adds structure around that span: candidate concept IDs, semantic type, section, part of speech, assertion values, medication details, or relations. Different components add different fields to the same in-memory note.
An extracted candidate is not a clinical conclusion
“No fever” can produce a fever concept row with negative polarity. That row says the extractor found the fever phrase and also found evidence that it was negated. It does not say the patient has fever. In the same way, a medication mention can describe an order, a plan, historical use, or another context depending on the note. The extracted row carries the evidence needed for interpretation; the row alone does not settle the clinical question.
| What the note says | What cTAKES can add | What still needs interpretation |
|---|---|---|
| “Right flank pain” | Phrase span, concept candidates, semantic type, assertion fields, and a location relation. | Whether the symptom supports a diagnosis in the study context. |
| “No fever” | A fever candidate with negative polarity. | Whether the negation applies correctly in the full note and time period. |
| “Start DemoMed 10 mg…” | Medication identity candidates plus dose, route, frequency, duration, and status change. | Whether the medication was prescribed, taken, stopped, reconciled, or merely discussed. |
46-62 and its candidate
the dictionary returns candidate meanings
The dictionary connects a phrase to candidate clinical meanings
“Fever,” “febrile,” and a supported abbreviation can refer to the same clinical idea even though their letters differ. A clinical terminology organizes those ideas as coded concepts. An ontology also records how concepts relate to one another. Those shared identifiers let two sites refer to the same candidate meaning without exchanging the original sentence.
When cTAKES reaches “fever,” it searches the local dictionary for spellings that match those characters. The returned rows are possible mappings into the selected terminology sources. They are candidates, not a promise that one code perfectly captures the sentence. The wrapper records which dictionary build was loaded because changing those searchable rows can change what the same sentence produces.
Terminology and ontology
UMLS is a metathesaurus that integrates many source vocabularies. It links synonymous or closely aligned source concepts under a CUI while preserving the original vocabulary and code. The source vocabularies may include ontologies with formal relationships, but UMLS itself should not be read as one perfect clinical ontology.
Dictionary
The dictionary is the local database that cTAKES searches while it processes a note. It contains the selected words, normalized forms, concept IDs, semantic types, and medication identifiers needed by this pipeline. It is built from a specific UMLS release and selected terminology sources.
The identifiers attached to the synthetic note
- CUI
- A UMLS concept identifier for a clinical meaning, represented here by fake values such as
DEMO-C002. - TUI
- A semantic-type identifier that says what kind of meaning the candidate represents, such as a sign or symptom.
- RxCUI
- An RxNorm identifier used for a normalized medication concept, represented by
DEMO-RX001in this example. - Preferred term
- The standard display name associated with the candidate even when the clinician used a synonym or abbreviation.
Dictionary selection changes what cTAKES can find. Loading every available term can add abbreviations and broad meanings that do not help the study, while filtering too aggressively can remove wording clinicians actually use. The USDHUB dictionary build selects the configured terminology sources and semantic types, audits the finished database, and gives the database an immutable release identity. The wrapper then checks that identity before the first note enters cTAKES. The current release is validated for controlled local testing; its source distribution decisions remain owner-review-required.
The exact source rules, UMLS tables, term-type filters, semantic-type set, build receipts, and local activation process are documented in Dictionary and recipes.
DEMO-C002 and DEMO-RX001
the recipe orders the components that touch them
The recipe decides which components touch the note
cTAKES is a collection of components, not one fixed extraction. A piper recipe lists the components that run and the order in which they run. That order changes the result because a later component can only use annotations that already exist.
The USDHUB recipe keeps one explicit path: language structure, dictionary candidates, default WSD, medication recognition, the five selected assertions, degree and location relations, and the requested writers. The filename and runtime checks sit outside cTAKES, so they surround this path without becoming clinical annotators themselves.
Sections and language structure give the words context
“Right flank pain” occurs under Assessment, while “Start DemoMed…” occurs under Plan. Section detection keeps those headings available. Sentence splitting, tokenization, part-of-speech tagging, and phrase chunking add the language boundaries used by later components. The original characters still do not move.
Dictionary lookup creates candidate meanings
The lookup component asks the active dictionary which concepts match each phrase. “Fever” can now carry a symptom candidate. A phrase with more than one dictionary match can carry several candidates on the same source span, which is where word-sense disambiguation enters the path.
Medication recognition adds the clinical details around the drug
The medication component treats “DemoMed 10 mg by mouth twice daily for 5 days” as more than a drug name. It can attach the dose, route, frequency, duration, and status-change phrase to the medication mention. Keeping the evidence text beside each normalized value lets a reviewer see how the field was obtained.
Assertions change how a concept row should be read
The concept “fever” is present in the characters, but “No” changes its polarity. Other assertion components can mark uncertainty, conditional language, generic statements, or the subject of the statement. A blank assertion value means the pipeline did not assign that attribute; it should not be silently read as a positive clinical fact.
Relations connect one annotation to another
“Right” and “flank” help locate the pain. A location relation carries that connection onto the symptom row, so the reviewer sees which location modified the pain. The recipe also supports degree relations where an indicator such as “severe” modifies a clinical mention.
The wrapper keeps the extraction connected to the run that made it
cTAKES handles the annotations inside one Java analysis. A usable batch also needs decisions before cTAKES starts, while it runs, and after it finishes. The wrapper owns those decisions so each site can size CPU, memory, and storage for its own machine while keeping the filename rule, dictionary, recipe, output shape, and version identity explicit.
The same synthetic note therefore has one continuous path: its filename becomes the document identity; the dictionary, normalized recipe, wrapper source, output schema, writer definition, and WSD state become the analysis identity; its annotations commit together; and the completed files are checked back against the source characters.
The filename carries the note identity
The HubID is the literal prefix USDHUB followed by exactly 20 digits.
The wrapper checks every filename and every HubID/NoteID pair before Java starts. That means an invalid name cannot make it halfway through a batch, and two files cannot quietly claim the same note identity.
The dictionary, recipe, and WSD profile are bound together
The active dictionary manifest names the UMLS release, selected database, and expected content hash. The normalized recipe names the component order. The WSD profile names the model and decision settings. The wrapper combines those inputs with the wrapper source, output schema, and writer definition, so a completed row can be traced back to the exact analysis definition that produced it.
The host has to fit the work before the work starts
cTAKES loads a Java virtual machine, the dictionary, and several models before it reaches the first note. The wrapper checks available CPU, memory, Linux commit capacity, output space, and temporary-work space before admitting the batch. Larger batches can be split across a safe number of Java processes, but each process still needs enough notes to justify loading the same models again.
One note is committed as one unit
The source note, document identity, concept rows, and medication rows are written in one SQLite transaction. If that note fails during the write, the database does not keep a partial set of rows that looks complete. After the batch closes, the wrapper reads the database and delivery files back and checks their document identities, source spans, evidence strings, and row values.
| Wrapper responsibility | What happens to the synthetic note | Downstream effect |
|---|---|---|
| Batch identity | The filename is parsed into one HubID and one NoteID before cTAKES starts. | The extracted rows can be joined back to the intended note without guessing from a path. |
| Analysis binding | The note runs with one recorded wrapper source, recipe, dictionary, WSD profile, and schema. | Another run can tell whether its extraction definition actually matched. |
| Capacity planning | The process starts only when the selected host can hold its models, work files, and result files. | A storage or memory failure is less likely to leave an incomplete batch. |
| Canonical write | The note and its rows land in SQLite as one transaction. | CSV and the viewer can come from the same completed result. |
| Read-back checks | “fever” must still match the recorded source span, and the exported cells must still match SQLite. | A completed marker represents checked files, not merely a Java exit code. |
The commands, protected-asset setup, batch run, and expected receipts are in the Getting started guide. The CPU, memory, storage, and shard calculations are in Startup and capacity.
WSD handles two meanings attached to the same words
Word-sense disambiguation, shortened to WSD, only has work to do when one source span has multiple candidate meanings. A phrase such as “mass” can refer to a body structure in one context and a quantitative concept in another. Dictionary lookup can attach both candidates to the same characters.
WSD looks at the nearby sentence, surrounding concepts, semantic types, preferred terms, and dictionary relationships. It can decide only when every candidate has a model profile, the leading candidate has direct lexical, graph, or preferred-term evidence, the leading raw score reaches the threshold, and the normalized margin clears the separation threshold. Both candidates remain in the result.
Same phrase, same two candidates
Change only the strength of the context evidence
Both rows remain. Complete profiles, direct evidence, score, and margin allow Y and N to be written.
The default USDHUB recipe runs WSD. A Y means that candidate won within that one ambiguous span. An N means another retained candidate on that same span won. Blank WSD fields mean there was no multi-candidate decision or the evidence was too weak. They do not mean the concept was absent.
The abstention is deliberate. Forcing a winner when two candidates are close hides uncertainty inside a clean-looking row. Keeping both candidates and leaving the WSD fields blank lets downstream rules decide whether to retain the ambiguity, send it to review, or exclude it from a particular analysis.
The exact evidence components, gates, model build, default behavior, and validation plan are explained in Word-sense disambiguation.
Y and N on span 170-174
each candidate becomes a source-linked SQLite row
Each result row stays connected to the words that produced it
The completed SQLite database keeps the original note text beside the concept and medication rows. Each row carries its document identity and character span, so the evidence can be read directly from the same stored note.
CSV files are exported from run-scoped SQLite views, so every delivery column comes from the canonical result. XMI is written in parallel when the complete native cTAKES structure is needed.
| Words in the note | Recorded span | Structured row | Evidence kept with the row |
|---|---|---|---|
| 46–62 | DEMO-C001, polarity POS | The exact phrase, Assessment section, candidate identity, assertion values, and the location “Right flank.” POS is an extractor assertion code, not proof of a diagnosis. | |
| 85–90 | DEMO-C002, polarity NEG | The exact phrase and the negative polarity created by “No fever.” | |
| 104–111 | DEMO-C003 / DEMO-RX001 | The medication identity plus “10 mg,” “by mouth,” “twice daily,” “5 days,” and “Start.” | |
| 170–174 | DEMO-C004A / Y and DEMO-C004B / N | The same source span keeps both candidates while WSD records which candidate won inside that ambiguity group. |
Files used for analysis
concepts.csv contains the requested concept, assertion, relation, and WSD fields. drugs.csv contains the requested medication identity and attribute fields. Both files come from SQLite views and are checked back against those views after export.
Files used for review
XMI keeps the native cTAKES structure for technical inspection. The clinician viewer combines the stored note and selected rows into one offline HTML file so a reviewer can move between a highlighted phrase and its structured fields.
Three identities answer three different questions
| Identity | What it combines | Question it answers |
|---|---|---|
| Release identity | Readable wrapper, dictionary, and cTAKES versions. | Which named software and terminology release family produced the result? |
| Analysis identity | Exact wrapper source, normalized recipe, dictionary database, output schema, writer definition, and WSD state. | Did two runs use the same extraction definition? |
| Run identity | One execution and its own logs and receipts. | Which local execution produced these files? |
Two executions should have different run identities even when they repeat the same analysis. Two sites should match the analysis identity before they treat their locally extracted rows as methodologically comparable. Keeping those questions separate prevents a timestamp from pretending to be a method version and prevents a readable version label from hiding a changed recipe or dictionary.
The field contracts, SQLite tables, CSV derivation, XMI role, version identities, receipts, and review boundary are documented in Outputs and review.
DEMO-C002
the reviewer opens it against the note
The reviewer can move from a row back to the note
The embedded viewer is built from the same synthetic SQLite result used in the examples. Select a concept or medication and the note keeps the source phrase visible beside the structured fields.
The review loop stays on one object: the row points to the source characters, the note shows the nearby language, and the field panel shows the assertion or medication attribute attached to that span.
A viewer made from a clinical batch contains the note text. It stays under the same access, encryption, transfer, and retention controls as the SQLite database and XMI files.
Synthetic clinician viewer
Hand-authored note, fake concept codes, and no patient data.
Matching local extraction comes before federated learning
Consider two hospitals studying kidney-stone care. Each hospital has its own notes, identifiers, security controls, and computing environment. The notes remain inside those environments.
Each hospital runs the same wrapper analysis locally, so “Right flank pain,” negated fever, medication attributes, and any study-specific concept features are created under the same recorded extraction definition. The source-linked rows remain local by default. A separate study protocol decides whether approved derived features, aggregates, or model updates may leave either site.
run_id A
run_id B
Only the exact payload named by the protocol leaves a site.
same analysis_id same extraction definition
different run_id different local execution
A concrete study path
Site A and Site B can each turn their own notes into the same local feature definition, for example a kidney-stone concept candidate in the Assessment section with non-negated polarity and a linked location. Each site can review that local feature against its own note text. Once both sites confirm the same analysis identity, a separate training process can use the approved feature locally and exchange only the permitted update or aggregate.
Federated learning adds machinery that this wrapper does not supply: a coordinating service, site authentication, a shared model definition, secure exchange or aggregation, failure handling, privacy accounting, and rules for what leaves each site. The wrapper records and checks one extraction definition at each site. Population, documentation, data-quality, and clinical-performance differences still require local review.
The longer example of local features, approved exchanges, secure aggregation, and the remaining study responsibilities is in Multi-site use.
Detailed guides
Install checksum-pinned dependencies, stage the protected dictionary and WSD files, validate filenames, run a batch, and read the receipts.
Review the clinical termsClinical basicsSpend more time with annotations, CUIs, RxCUIs, assertions, relations, medication fields, WSD, and the clinical interpretation boundary.
Inspect terminology choicesDictionary and recipesSee which UMLS sources and semantic types enter the dictionary, how the database is built, and why recipe order changes the result.
Inspect the controlsWrapper detailsFollow filename validation, analysis binding, canonical writes, source-span checks, result identities, and the offline viewer.
Plan a real batchStartup and capacitySee how Java startup, models, CPU, memory, commit, storage, and async shards affect a run.
Review a completed resultOutputs and reviewRead the SQLite schema, CSV fields, XMI role, version identities, sanity receipts, and clinical-data handling rules.