INTELLIGIBILITY SEVERITY RATING OF FRENCHAY SINGLE …



INTELLIGIBILITY SEVERITY RATING OF FRENCHAY SINGLE WORDS AND SENTENCES BY NAÏVE LISTENERS

SUBJECTS

Session 1

The raters were 5 health service professionals (4 females : 1 male) with no history of hearing impairment.

None of the raters were familiar with the individuals from whom the speech samples had been taken, nor with the test items.

Session 2

The raters were 5 health service professionals (5 females) with no history of hearing impairment.

None of the raters were familiar with the individuals from whom the speech samples had been taken, nor with the test items.

EQUIPMENT

Session 1

Written introduction and instructions for the rating procedure (see Appendix 1).

Toshiba Satellite Pro 4300 series laptop computer.

Opus Technology multimedia speaker system SP-699.

Wave Lab Lite playback facility.

Session 2

Written introduction and instructions for the rating procedure (see Appendix 1).

Toshiba Satellite 4090 CDS series laptop computer.

Opus Technology multimedia speaker system SP-699.

Microsoft Media Player.

SPEECH SAMPLE

Session 1/2

10 single words randomly generated from the word list of the Frenchay Dysarthria Assessment.

10 sentences randomly generated from the sentence list of the Frenchay Dysarthria Assessment.

Picture description (Appendix 2).

METHOD

Session 1/2

Rating took place within a quiet room on health authority premises. Each subject was given a copy of the introduction, instructions and transcription forms. Subjects were given the opportunity to ask questions prior to the starting of the transcription exercise. It was stressed that the rating exercise was not a ‘test’ of their ability to decipher the speech sample and that there should be no conferring once the speech samples were being played.

Subjects were first played an example speech sample of 2 single words, 2 simple sentences (carrier phrase plus changing verb), and a longer sentence to illustrate connected speech. This example was spoken by a speaker with no history of dysarthria. The example was transcribed to act as a lead-in exercise, to check on the quality of the play back facility and to check on the inter-rater reliability of transcribing normal speech.

Following the example transcription, subjects were given the opportunity to ask for clarification of the requirements of the transcription. The dysarthric speech samples were then played in the order single words, sentences, picture description. A short pause was left between each item to allow for transcription. Each set of samples took approximately 10 minutes to complete per speaker.

At the end of the transcription the completed forms were collected by the examiner and responses tabulated.

RESULTS

Tables have only been completed for single words and sentences. For tables of individual responses, see Appendix 3.

Summary

| |SUBJECT |

| |CC |PH |GR |JP |JT |

|Single word |Range |0–20% |0-50% |0 |0-10% |10-40% |

| |Mode |10% |NA |0 |0 |20% |

| |Mean |10% |22% |0 |4% |22% |

|Sentences |Range |0-10% |20-60% |0 |0-10% |0-20% |

| |Mode |10% |30% |0 |0 |10% |

| |Mean |10% |34% |0 |2% |10% |

COMMENTS

All subjects fall within the severe range on judgements of intelligibilty. PH scored better on the sentences but largely due to the higher recognition rate of one rater.

Picture descripton has not been analysed at this time. Raters were asked to note down any specific words or phrases that they could discern. Analysis was initially to be made on the number of recognisable words as a percentage of the total number of words in the sample. However, the spectrum of recognition was too broad across the group to make a valid judgement. In the second session, raters were asked to score the picture description according to the Frenchay test intelligibility judgements for connected speech. None of the subjects were rated above the level of ‘Ocasional words decipherable’. The majority of ratings were at the ‘Totally unintelligible’ level.

25/04/02

MP

Appendix 1

STARDUST Project -Intelligibility Severity Rating – Naïve Listener

Frenchay – Single words and sentences

Connected Speech (Picture Description)

INTRODUCTION

STARDUST is a Department of Health funded project being run between the University of Sheffield and the Medical Physics Department of Barnsley District General Hospital. The aim of the project is to develop a new voice recognition system that can be used by people with severe physical disability and severe speech problems (dysarthria) to access an environmental control system. This voice recogniser would supplement or even replace the time consuming and often effortful scanning/switching system that many people have to use to access environmental controls.

Voice recognition systems have been developed for people with mild speech production problems. This project is aimed at those people with severer problems than are frequently reported in the literature. As part of the project it is necessary to rate the severity of the speech problem in terms of ‘intelligibility’. Intelligibility is actually quite difficult to judge. People very quickly tune in to different speech, so getting a ‘real’ rating of how easy somebody is to understand is difficult if you rely on people who are familiar with that person’s way of speaking. We are therefore asking people who have never met the volunteers who have agreed to take part in this research to listen to a series of words, sentences and a short passage ofconnected speech, to see how much they can understand.

WHAT AM I BEING ASKED TO DO?

You will shortly be asked to listen to 10 words. Each word will be played only once, with a short pause to allow you to write down what you think you have heard the person say. If you could not understand the word at all, just write ‘Don’t know’, otherwise write down exactly what you thought you heard.

After the single words you will hear 10 short sentences. Each sentence will be played only once. The first part of the sentence is the same each time. Only the last word will change. As before, please write down exactly what you thought you heard. If you could not understand at all, please write ‘Don’t know’.

The final part is a short passage of connected speech, based on a description of a picture. You will not be shown the picture. Please just write down any words or phrases that you understood the person to say.

When you are listening to the items please do NOT confer with your neighbour about what you thought the word or phrase might have been. It is very important in terms of rating the ‘severity’ of the speech that the responses that we receive are your own interpretation of what you heard. It is just as important that we know what you thought the person said, as much as getting a correct answer.

The information that you provide is for group analysis only. You do not have to write your name on any of the sheets.

If you have any questions, please ask them before the items are played.

Thank you for your time.

STARDUST – INTELLIGIBILITY SEVERITY RATING

Date:

Location:

EXAMPLE

Single Words

1.

2.

Sentences

1.

2.

Connected Speech

SUBJECT IDENTIFIER : GR

Single Words

1. 6.

2. 7.

3. 8.

4. 9.

5. 10.

Sentences

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

Connected Speech Sample

Intelligibility Rating

(Please circle the statement letter that you feel applies to this speech example)

a. No abnormality

b. Speech abnormal but intelligible – occasionally has to repeat

c. Speech severely distorted, can be understood half the time. Very often has to repeat

d. Occasional words decipherable

e. Totally unintelligible

Appendix 2

Line drawing of a flood scene.

Appendix 3

PRE RATING TEST EXAMPLES

|Item No. |Target word |Rater 1 |Rater 2 |Rater 3 |Rater 4 |Rater 5 |

|SINGLE WORD |CAT |Cat |Cat |Cat |Pat |Cat |

|SINGLE WORD |TABLE |Table |Table |Table |Table |Table |

|SENTENCE |THE MAN IS WALKING |Walking |Walking |Walking |Walking |Walking |

|SENTENCE |THE MAN IS WASHING |Washing |Washing |Washing |Washing |washing |

|CONNECTED SPEECH |THE CAT IS SITTING ON |The cat is |The cat is sitting |The cat is sitting |The cat is sitting |The cat is sitting |

| |THE TABLE AND WASHING |sitting on the |on the table and |on the table and |on the table and |on the table and |

| |ITS PAWS. |table and washing|washing its paws |washing its paws |washing its paws |washing its paws |

| | |its paws | | | | |

|Total Items |15 |15 |15 |15 |14 |15 |

|Total % Identified | |100 |100 |100 |93 |100 |

|Mean Identification |98% | |

Intelligibility Testing – Frenchay Single words

Subject – CC

|Item No. |Target word |Rater 1 |Rater 2 |Rater 3 |Rater 4 |Rater 5 |

|9 |JACKET |DK |DK |DK |DK |DK |

|13 |GROW |Girl |Gloves |Go |Clothes |DK |

|15 |BUBBLE |Gurgle |Bottle |DK |DK |DK |

|16 |STUBBLE |DK |Sugar |Sugar |Table |Sugar |

|18 |DOUBLE |Dribble |DK |Football |Table |DK |

|22 |AIR |Air |Lair |Air |Air |Air |

|26 |SPRINKLE |Twinkle |DK |DK |DK |DK |

|49 |FAT |Cut |Cat |Cat |Tap |Top |

|5 |PARK |Hack |DK |Clock |Black |Park |

|2 |SWARM |Fall |DK |Room |Wand |Warm |

|Total Items |10 |1 |0 |1 |1 |2 |

|Total % Identified | |10 |0 |10 |10 |20 |

|Mean Identification |10% | |

Severity rating – SEVERE

Intelligibility Testing – Frenchay Sentences

Subject – CC

|Item No. |Target word |Rater 1 |Rater 2 |Rater 3 |Rater 4 |Rater 5 |

|43 |DANCING |DK |Standing |DK |Dancing |DK |

|11 |RECEDING |DK |DK |DK |DK |DK |

|22 |TICKING |Sitting |Kicking |Kicking |Kicking |DK |

|25 |BLOCKING |DK |Broken |DK |Looking |Broken |

|29 |CATCHING |Catching |DK |DK |Watching |DK |

|30 |PITCHING |DK |DK |DK |DK |DK |

|33 |PINNING |DK |DK |DK |Punching |DK |

|39 |STANDING |Chairing |DK |Standing |Dancing |DK |

|40 |BENDING |DK |DK |DK |DK |Selfish |

|4 |CREEPING |DK |DK |Crying |Catching |Crying |

|Total Items |10 |1 |0 |1 |1 |0 |

|Total % Identified | |10 |0 |10 |10 |0 |

|Mean Identification |6% | |

Severity rating – SEVERE

Intelligibility Testing – Frenchay Single Words

Subject – PH

|Item No. |Target word |Rater 1 |Rater 2 |Rater 3 |Rater 4 |Rater 5 |

|47 |LAIR |Rare |Glare |There |Lair |Air |

|9 |JACKET |DK |DK |DK |Turkey |Jackie |

|20 |HERE |Year |Year |Here |Here |Year |

|21 |ERROR |DK |Everyone |Everyone |DK |Table |

|25 |CYCLE |Cycle |Side door |Cycle |Cycle |Side draw |

|26 |SPRINKLE |Sprinkle |DK |Twinkle |Sprinkle |DK |

|29 |PLAY |Parade |DK |Away |Play |DK |

|42 |THOUGHT |DK |Haunt |Font |DK |DK |

|46 |LOYAL |Loyal |DK |Loyal |Royal |Lion |

|5 |PARK |DK |DK |Book |DK |Pane |

|Total Items |10 |3 |0 |3 |5 |0 |

|Total % Identified | |30 |0 |30 |50 |0 |

|Mean Identification |22 | |

Severity rating – SEVERE

Intelligibility Testing – Frenchay Sentences

Subject – PH

|Item No. |Target word |Rater 1 |Rater 2 |Rater 3 |Rater 4 |Rater 5 |

|46 |TUCKING |Talking |Talking |Talking |Talking |Talking |

|5 |PATCHING |DK |Batting |Watching |Catching |DK |

|14 |DAMNING |Damning |DK |Thinking |DK |DK |

|22 |TICKING |Ticking |Ticking |Thinking |Ticking |DK |

|24 |AIRING |Airing |DK |Hearing |Hearing |DK |

|29 |CATCHING |Catching |Catching |Catching |Catching |Catching |

|37 |DRINKING |Drinking |Ringing |Drinking |Ringing |DK |

|39 |STANDING |Standing |Standing |Standing |Standing |Standing |

|45 |TUGGING |DK |DK |Talking |Talking |DK |

|3 |CREEPING |Cleaning |Screaming |Screaming |Screaming |Screaming |

|Total Items |10 |6 |3 |3 |3 |2 |

|Total % Identified | |60 |30 |30 |30 |20 |

|Mean Identification |34% | |

Severity rating – SEVERE – (moderate)

PRE RATING TEST EXAMPLES

|Item No. |Target word |Rater 1 |Rater 2 |Rater 3 |Rater 4 |Rater 5 |

|SINGLE WORD |CAT |Cat |Cat |Cat |Pat |Cat |

|SINGLE WORD |TABLE |Table |Table |Table |Table |Table |

|SENTENCE |THE MAN IS WALKING |Walking |Walking |Walking |Walking |Walking |

|SENTENCE |THE MAN IS WASHING |Washing |Washing |Washing |Washing |washing |

|CONNECTED SPEECH |THE CAT IS SITTING ON |The cat is |The cat is sitting |The cat is sitting |The cat is sitting |The cat is sitting |

| |THE TABLE AND WASHING |sitting on the |on the table and |on the table and |on the table and |on the table and |

| |ITS PAWS. |table and washing|washing its paws |washing its paws |washing its paws |washing its paws |

| | |its paws | | | | |

|Total Items |15 |15 |15 |15 |14 |15 |

|Total % Identified | |100 |100 |100 |93 |100 |

|Mean Identification |98% | |

Intelligibility Testing – Frenchay Single words

Subject – GR

|Item No. |Target word |Rater 1 |Rater 2 |Rater 3 |Rater 4 |Rater 5 |

|10 |JAGGED |DK |DK |DINNER |BURGER |DK |

|11 |GLOW |DK |DK |BALL |GO |BUSINESS |

|25 |CYCLE |DK |DK |SUGAR |DIGGER |DK |

|30 |PAY |DK |DK |DK |BALL |DK |

|31 |BRIAR |DK |DK |DK |DK |DK |

|37 |THOUGH |DK |DK |TEA |OK |DK |

|39 |URGENT |DK |DK |AGAIN |DK |GONE |

|40 |SERGEANT |DK |DK |BIGGER |DIGGER |BETTER |

|44 |THORN |GO |GO |DONE |GO |DK |

|49 |FAT |DK |DK |DK |FARMER |DK |

|Total Items |10 |0 |0 |0 |0 |0 |

|Total % Identified | |0 |0 |0 |0 |0 |

|Mean Identification |0 | |

Severity rating – SEVERE

Intelligibility Testing – Frenchay Sentences

Subject – GR

|Item No. |Target word |Rater 1 |Rater 2 |Rater 3 |Rater 4 |Rater 5 |

|03 |CREEPING |DK |GOING |DK |GOING |GOING |

|06 |COMING |DK |DK |GOING |DK |DK |

|08 |SPINNING |BIGGER |RUNNING |LEAVING |BIGGER |DK |

|13 |TINKERING |DK |DK |DK |DK |DK |

|19 |PRANCING |BIGGER |DK |DK |DK |DK |

|25 |BLOCKING |BIGGER |BIGGER |BIGGER |BIGGER |BIGGER |

|32 |THUMBING |DK |DK |DIGGING |DIGGING |DK |

|36 |NAMING |NEARER |DK |SMALLER |LENDING |NERVOUS |

|46 |TUCKING |DK |DK |KICKING |GIDDY |DK |

|48 |GAGGING |BIGGER |DK |DK |GINGER |DK |

|Total Items |10 | | | | | |

|Total % Identified | |0 |0 |0 |0 |0 |

|Mean Identification |0 | |

Severity rating – SEVERE

Intelligibility Testing – Frenchay Single words

Subject – JP

|Item No. |Target word |Rater 1 |Rater 2 |Rater 3 |Rater 4 |Rater 5 |

|04 |SPARK |WORK |LOOK |LOOK |LOOK |DRUNK |

|12 |GO |GO |DK |DK |GO |GONE |

|13 |GROW |NO |DK |DK |ROAD |BLOW |

|15 |BUBBLE |BOOK |DK |BOOK |BOOK |DK |

|22 |AIR |HAIR |HAIR |HAIR |HAIR |HAIR |

|24 |JUNGLE |DK |DK |DK |DK |DK |

|25 |CYCLE |CAKE |DK |DK |CUP |TAKE |

|32 |PRIOR |CAKE |DK |CRY |CRY |PLAY |

|41 |BROUGHT |WORK |DK |DK |WORK |WORK |

|43 |BRAWN |WORK |NICE |WORK |BOOK |DK |

|Total Items |10 |1 | | |1 | |

|Total % Identified | |10% |0 |0 |10% |0 |

|Mean Identification |4% | |

Severity rating – SEVERE

Intelligibility Testing – Frenchay Sentences

Subject – JP

|Item No. |Target word |Rater 1 |Rater 2 |Rater 3 |Rater 4 |Rater 5 |

|13 |TINKERING |DK |DK |DK |DK |DK |

|29 |CATCHING |DK |DK |DK |CATCHING |DK |

|30 |PITCHING |SPEAKING |DK |SLEEPING |DK |DK |

|33 |PINNING |BIG |DK |PLAYING |PLAYING |DK |

|34 |FINDING |VERY BIG |DK |HIDING |DK |HIDING |

|40 |BENDING |READING |DK |DK |DK |DK |

|41 |BRINGING |DK |DK |DK |DK |DK |

|42 |BITING |DK |DK |WASHING |WATCHING |DK |

|43 |DANCING |MAKING A SHOE |DK |DK |LOOKING |DK |

|47 |PICKING |SPEAKING |PAINTING |KICKING |PICKING |DK |

|Total Items |10 | | | | | |

|Total % Identified | |0 |0 |0 |10% |0 |

|Mean Identification |2% | |

Severity rating – SEVERE

Intelligibility Testing – Frenchay Single words

Subject – JT

|Item No. |Target word |Rater 1 |Rater 2 |Rater 3 |Rater 4 |Rater 5 |

|12 |GO |GO |GO |GO |GO |GO |

|14 |GOAT |GOAT |GOOD |READ |GOAT |DK |

|26 |SPRINKLE |PEACOCK |DK |SWEETCORN |THANKYOU |BECAUSE |

|28 |SLAY |SAY |DK |PLAY |SLAY |DK |

|30 |PAY |PLAY |GET |DAY |PAY |BIRD |

|31 |BRIAR |BEAR |BEAR |PAY |BEER |DK |

|32 |PRIOR |FIRE |DK |HAY |DK |DK |

|33 |AREA |AIR |HAIR |DK |A |DK |

|34 |FLOOR |FOUR |FLOOR |FOUR |FOR |FLOOR |

|41 |BROUGHT |BOAT |WOOD |BOAT |BOAT |BOAT |

|Total Items |10 |2 |2 |1 |4 |2 |

|Total % Identified | |20% |20% |10% |40% |20% |

|Mean Identification |22% | |

Severity rating – SEVERE

Intelligibility Testing – Frenchay Sentences

Subject – JT

|Item No. |Target word |Rater 1 |Rater 2 |Rater 3 |Rater 4 |Rater 5 |

|06 |COMING |COMING |COMING |COMING |DK |DK |

|13 |TINKERING |SLEEPING |DK |SPEAKING |THINKING |SPEAKING |

|14 |DAMNING |BUSY |GOING |GOING |HEAVY |GOING |

|24 |AIRING |DK |DK |DK |HEAVY |DK |

|25 |BLOCKING |DK |DK |BATTING |MOPING |DK |

|28 |REAPING |READING |DK |READING |DK |READING |

|31 |NUMBING |DK |DK |DK |DK |DK |

|43 |DANCING |DK |DK |BENDING |BENDING |DK |

|44 |FENCING |DK |DK |THINKING |BENDING |DK |

|49 |HEARING |EATING |DK |HEARING |HEARING |DK |

|Total Items |10 |1 |1 |2 |1 |0 |

|Total % Identified | |10% |10% |20% |10% |0 |

|Mean Identification |10% | |

Severity rating – SEVERE

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download