2010 i2b2 / VA Challenge Evaluation



2010 i2b2 / VA Challenge Evaluation

Annotation Guidelines for Concepts v9

Ozlem Uzuner and Scott DuVall

The concept extraction task builds towards relation extraction task of the challenge. Therefore, from the wealth of concept and concept types that are found in clinical records, only a small subset that directly contribute to the relation challenge have been identified.

This guideline describes the specific types of information that should be annotated for the concept extraction task and provides examples similar to those that may be found in the clinical text. The instances that should be marked along with examples of the surrounding text that should be included in the annotation are described. Instances in this guideline marked in “BLUE” are correctly annotated concepts. Instances marked in RED are those that should not be marked.

General Guidelines for Concepts

Syntactic forms of concepts:

Only complete noun phrases (NPs) and adjective phrases (APs) should be marked.

▪ The man was [obese]. The obese man came to the clinic.

▪ Patient underwent catheterization. Catheterization report showed …

Include all modifiers with concepts when they appear in the same phrase except for assertion modifiers. (Please see assertion annotation guidelines for a description).

▪ “bilateral DVT”

▪ “recurrent angina”

▪ “high grade LAD lesion”

▪ “chronic hepatitis”

▪ “IV Ciprofloxacin”

▪ “cataract surgery”

▪ “diabetes medication”

▪ “head CT”

▪ “chest X-ray”

▪ no “fever”

▪ possible “tamponade”

Include the first prepositional phrase (PP) following an annotated instance only when the PP indicates either an organ/body part or when the PP and NP can be rearranged to eliminate the PP (we later call this the PP test).

▪ “lysis of adhesion” -> adhesion lysis

▪ “repair of ventral hernia”-> ventral hernia repair

▪ “placement of stent” -> stent placement

▪ removal of mass from surface of transverse colon -> mass removal from surface of transverse colon (only include the first PP)

Include articles and possessive pronouns

▪ his cancer

▪ her medications

▪ the surgery

▪ an appendectomy

▪ patient underwent resection of his tumor

Semantic categories of concepts:

Concepts are defined in three general semantic categories that are each annotated separately.

1) Medical Problems: phrases that contain observations made by patients or clinicians about the patient’s body or mind that are thought to be abnormal or caused by a disease. They are loosely based on the UMLS semantic types of pathologic functions, disease or syndrome, mental or behavioral dysfunction, cell or molecular dysfunction, congenital abnormality, acquired abnormality, injury or poisoning, anatomic abnormality, neoplastic process, virus/bacterium, sign or symptom, but are not limited by UMLS coverage.

What should be annotated?

➢ Medical problems need to be things that are wrong with the patient and can be treated.

➢ They need to be complete noun phrases (NPs) or adjective phrases (APs).

➢ They need to be able to be modified with a negation modifier.

➢ They need to belong to one of the above semantic types.

All four of these conditions have to hold.

Examples of Medical Problems to annotate:

a) Phrases that name a disease, syndrome, sign, or symptom

▪ Ativan 0.25 to 0.5 mg IV q 4 to 6 hours prn anxiety

o What was treated? anxiety. (NP, can be negated with “no anxiety”)

▪ the wound was noted to be clean with mild serous drainage

o Question test: What was treated? The wound.

o Question test: What was treated? mild serous drainage.

o Question test: What was treated? Clean. ( wrong.

▪ the tumor was growing

o Question test: What was treated? The tumor.

▪ an echocardiogram revealed a pericardial effusion and possible tamponade clinically .

▪ The patient presented with the symptoms of menorrhagia .

▪ elevated enzymes resolved

▪ the drop in hematocrit was secondary to his

▪ the patient has had increasing dyspnea on exertion and increasing weight gain to the tune of approximately 30 lbs despite increasing diuresis

▪ hematocrit drop was secondary to his…

▪ Includes atrial fibrillation with good heart rate control on Digoxin

▪ enzymes mildly elevated but not actively ischemic

▪ dropping hematocrit was secondary to his …

▪ November 19 , 1997 , he underwent a resection of a skull base chordoma and a lateral mandibulotomy with tracheostomy .

▪ Catheterization report in 1988 showed subtotal occlusion of the RCA

▪ Cath report showed 50% ulcerated LAD

b) Mental or behavioral status observations

▪ His mental status changes remained stable

▪ the patient developed a sudden change in her mental status

o PP test “a sudden change in her mental status”

▪ she did well except for some episodes of confusion

o PP test “some confusion episodes”

c) Virus and bacterium

▪ Blood cultures were positive for S. Veridans

d) Injury

▪ patient arrived with a broken arm

e) Abnormalities

▪ The defects were found

▪ chest x-ray revealed an abnormality

Examples of concepts that should not be marked as Medical Problems:

a) Statements about normal states of health

▪ she had been comfortable on the Morphine drip

▪ wound was clean

▪ patient was stable , afebrile

▪ He was dynamically stable on his Dobutamine

▪ converted back to normal sinus rhythm

▪ was converted to sinus rhythm with Digoxin loading

▪ his hematocrit remained very stable

▪ serum uric acid was normal

b) Physiologic measurements, vital signs, or the tests that measure them and statements of bodily function (even if the measurement could be outside normal range)

▪ bowel movements increased

▪ EF 45%

▪ Blood pressure 145/95

▪ serum uric acid was normal

▪ will monitor BUN / Cr after giving Lasix

▪ the patient 's respiratory status continued to decline

o What was treated? Respiratory status ( wrong

▪ hematocrit dropped

o what’s treated? Hematocrit ( wrong

▪ his hematocrit remained very stable

▪ baseline CRI (1.2-1.6) with elevation at admission probably prerenal

▪ the patient 's LDH climbed

c) Verbs that describe the outcome of an event

▪ the patient defervesced

▪ pneumonia appeared to resolve

▪ the tumor was growing

▪ the patient 's respiratory status continued to decline

▪ enzymes mildly elevated but not actively ischemic

o enzymes mildly elevated ( not an NP

▪ hematocrit dropped

▪ Her ileostomy was pink and functioning

▪ Her incisions were healing well

▪ Oxygenation improved with Dobutamine

▪ Oxygenating really well while intubated

▪ He was hemodynamically stable

▪ He was extubated successfully and was breathing comfortably on his own

▪ He was dynamically stable on his Dobutamine with improved oxygenation.

▪ Her blood pressure medications were held to allow increased cerebral perfusion.

d) Partial noun phrases

▪ He was a moderately obese man in acute respiratory distress .

2) Treatments correspond to UMLS semantic types therapeutic or preventive procedure, medical device, steroid, pharmacologic substance, biomedical or dental material, antibiotic, clinical drug, and drug delivery device. Other concepts that are treatments but that may not be found in UMLS are also included.

What should be annotated?

➢ In general, the annotated concepts need to be things that can treat a problem. They need to sensibly answer the question “what fixed the medical problem?”

➢ They need to be complete noun phrases (NPs) or adjective phrases (APs).

➢ They need to belong to one of the above semantic types.

All three of these conditions have to hold.

Examples of Treatments to annotate:

a) Medication names, brand names and generics as well as collective names for groups of medications

▪ Gentamicin

▪ Lactulose

▪ Lasix

▪ calcium carbonate

▪ ceftizoxime

▪ IV Ciprofloxacin

o Include mode

▪ IV heparin

▪ oral medication

▪ p.o. kay ciel

▪ regular sliding scale insulin

▪ Vitamin K therapy

▪ Vivonex T.E.N.

▪ med therapy

▪ sedation

▪ statin

▪ Patient was on morphine drip

▪ and received induction chemotherapy

b) Biological substances

▪ creatinine improved with hydration and the packed red blood cell transfusions

▪ the patient remained on IV hydration therapy

▪ lipids , ASA, statin

▪ he did not require a transfusion

c) Drug and treatment delivery devices

▪ ICD shocks

▪ he was responsive to having an NG feeding tube placed with Jevity feeds started

▪ were unable to maintain oxygen saturations above 80% even using reverse I to E ventilation , paralysis , sedation , and other modes of ventilation

▪ the patient was stable from a respiratory standpoint using her respirator at night .

▪ the patient remained hemodynamically stable on the ventilator

d) Procedures and devices or hardware involved in those procedures

▪ the patient had a bronchoalveolar lavage performed

▪ sigmoid colon was noticed at the time of operation

▪ history of a low anterior resection in May of 1998

▪ the needle jejunostomy tube was utilized on the first postoperative day

▪ After tube removal, she was started on…

o Nominal form of the verb is to be included

▪ Significant for radiation therapy after his surgery in July

▪ the staples were removed

▪ The sheaths were removed on 11/4/92 and heparin was restarted four hours after sheath removal

▪ status post myocardial infarction in 1989 with stent placement

▪ who comes in for placement of an Omaya reservoir

o PP test: an Omaya reservoir placement

▪ After removal of the intra-aortic balloon pump the patient was…

o PP test: the intra-aortic balloon pump removal

▪ by the time the cytologist came to do a fine needle aspiration the mass had shrunken to the point where there was nothing to aspirate

▪ No axillary dissection was done

▪ Total abdominal hysterectomy, bilateral salpingooophorectomy, bilateral lymph node dissection, omentectomy, lysis of adhesions, removal of mass from surface of transverse colon

▪ Physical Therapy

▪ repair of ventral hernia, and needle jejunostomy tube placement.

▪ The patient was evaluated for repair of false femoral aneurysm

▪ Significant for radiation therapy after his surgery in July , a tracheoesophageal fistula , which was repaired , exploratory laparotomy in November , 1994 , for small bowel obstruction, … drainage of a neck abscess

▪ November 19 , 1997 , he underwent a resection of a skull base chordoma and a lateral mandibulotomy with tracheostomy .

Examples of concepts that should not be marked as Treatments:

a) Verbs that indicate application of a treatment

▪ Her dressing was changed

▪ …had PPM and AICD placed

▪ Tube was removed

▪ He was hypotensive and intubated at that time

▪ In 1984 , he had the right knee aspirated

▪ The sheaths were removed on 11/4/92 and heparin was restarted four hours after sheath removal

▪ …was treated successfully with bagging and tracheal suctioning

▪ her varices were banded

▪ this was subsequently repaired .

▪ by the time the cytologist came to do a fine needle aspiration the mass had shrunken to the point where there was nothing to aspirate

▪ Significant for radiation therapy after his surgery in July , a tracheoesophageal fistula , which was repaired , exploratory laparotomy in November , 1994 , for small bowel obstruction, … drainage of a neck abscess

b) Partial noun phrases

▪ arrived in surgery suite

▪ the needle jejunostomy tube was utilized on the first postoperative day

3) Tests correspond to UMLS semantic types laboratory procedure, diagnostic procedure, and instances not covered by UMLS.

What should be annotated?

➢ In general, the annotated concepts need to be things that help determine whether the patient has a problem. Tests are analogous to treatments, but instead of fixing the problem, they are used to find more information out about a problem. They need to sensibly answer the question “what was done to find out if there was a medical problem?”

➢ They need to be complete NPs or APs.

➢ They need to belong to one of the above semantic types.

All three of these conditions have to hold.

Examples of Tests to annotate:

a) Procedures performed on the patient

▪ An abdominal ultrasound was performed showing no stones .

▪ Chest x-ray revealed clear lungs

▪ A head computerized tomography scan was planned to evaluate his change in mental status

▪ Cardiac catheterization at that time revealed proximal and distal 50% right coronary artery lesions

▪ An echocardiogram revealed an ejection fraction of 25%

▪ Chest x-ray showed bilateral pneumothorax

▪ A lung biopsy was performed , revealing chorio carcinoma pathologically

▪ EKG revealed normal sinus rhythm with no ST changes .

▪ A mediastinoscopy was also performed at that time .

▪ However , a VQ scan was performed , which revealed intermediate probability .

b) Panels and tests run on patient body fluids

▪ His urinalysis showed 10-20 granular casts

▪ CBC was unremarkable .

▪ Metabolic 20 panel was unremarkable .

▪ Blood cultures were positive for S. Veridans

▪ HCT : 30.1

▪ WBC : 12.8

▪ CRE : 1.2

c) Physiologic measures and vital signs

▪ Blood pressure 120/80

▪ Pulse : 40

d) Physical examination of the patient

▪ Cardiac exam revealed an irregular rate

▪ Rectal exam was heme negative

▪ rash apparent upon evaluation

Examples of concepts that should not be marked as Tests:

a) Verbs that indicate application of a treatment

▪ The patient was evaluated for repair of false femoral aneurysm

▪ A lung biopsy was performed , revealing chorio carcinoma pathologically

▪ EKG revealed normal sinus rhythm with no ST changes .

▪ A mediastinoscopy was also performed at that time .

▪ However , a VQ scan was performed , which revealed intermediate probability .

b) Partial noun phrases

▪ Catheterization report in 1988 showed subtotal occlusion of the RCA

▪ Cath report showed 50% ulcerated LAD

c) Test values and measurements

▪ Blood pressure 120/80

▪ Pulse : 40

▪ HCT : 30.1

▪ a VQ scan was performed , which revealed intermediate probability

d) Mentions of tests that are stated as problems (these are marked as problems)

▪ The patient has high blood pressure

▪ elevated enzymes were noted

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download