To:



Evaluation of Human Computer Literacy Through Recognition of Lexical Emotional Significance

David Dvorak and Amy Horan

PROBLEM

Many people just see computers as machines. Despite the fact that some are able to make very intelligent computations and decisions, they still make these decisions by only using logic. It has been suggested that until artificial intelligence agents can utilize or recognize emotion, they will never be anything more than elaborate machines that rely on logic to keep up the appearance of intelligence. In terms of Star Trek: although Mr. Spock is the intelligent, logical character, it’s the emotionally charged Captain Kirk who always makes the decisions that save his life along with the lives of his crew. Many have tried to encapsulate the essence of human intelligence within software, but the one crucial human element that has been overlooked is emotion. For an agent to be perceived as intelligent, it must have a distinct personality like a human. This emotional aspect is the key to turning a simple machine into an emotional entity. A machine cannot simply be trained to perform an algorithm, it has to be trained to learn the emotional significance of human communication.

RELATED WORK

At Carnegie Mellon University, the Social Robots Project has produced a robot named Vikia. Vikia has a distinctly developed personality that is suited for casual conversation (CMU Social Robots Project, 2003). Through speech and visual recognition sensors, Vikia is able to carry on dialogue and is hoped to bridge the gap between human and robot interaction. Eventually, the team plans to teach the robot human conventions to help with its interaction process. Another relevant application of emotion to artificial agents is the robot Kismet. Kismet is designed to interact comfortably with humans by producing distinct facial expressions to convey its mood (Social Machines – Overview, 2003). Both of these robots demonstrate that recognition of and response to emotion are reasonable goals within the field of artificial intelligence. They also demonstrate that it is possible to build an agent that a human will feel comfortable with. If a human can interact with a robot comfortably, then this technology can be applied in a variety of useful ways. As far as our project goes, computer companies could develop intelligent programs for the purposes of customer service. A program like ours could be used to measure a person's level of computer literacy, then the agent could help the customer without confusing the person by talking beyond the customer's level of comprehension. The bottom line is that agents can only be said to possess human intelligence once humans feel as comfortable with the agent as they would with any other human.

Natural language processing is a huge part of building a successful conversational agent, however some would consider it a slow moving field. According to Robby Garner, a former Loebner prizewinner, grammar isn't as important to natural language processing as it would seem. According to Garner in 2003, "we must be able to define "understanding" and "meaning" and be able to integrate that with behavior that is purposeful, and understandable." So, natural language processing isn't simply picking nouns and other parts of speech out of a sentence. It entails grabbing the bits of information that are useful to your agent and then integrating them into the program so that they become useful. Jurafsky and Martin have a more specific definition of natural language processing. They claim that to utilize and capitalize on language knowledge, six categories must be focused on. These categories are morphology (the study of meaningful components in a sentence), syntax (structural relationships between words), semantics (the study of meaning), pragmatics (how language is used to accomplish goals), discourse (large linguistic units), and phonetics (Jurafsky and Martin, 2000). Phonetics will not be relevant to us, given that our program will not utilize audio components. Another important aspect of dissecting language is disambiguation. Language can contain many ambiguities, so part of natural language processing is pinpointing the precise meaning of a phrase when it has the potential to have multiple meanings. Jurafsky and Martin discuss many algorithms and models that are necessary to work with language. State machines, formal rule systems, and search space algorithms are all critical components for interpreting sentences. Search algorithms, with the help of state machines and formal rule systems, search through a space state composed of hypotheses about input to extract meaning from language (Jurafsky and Martin, 2000).

OBJECTIVES

There have been many developments in the integration of emotion into artificial intelligence. Our objective in this project is to gain a better understanding of how emotion factors into artificial intelligence by writing a minimal conversational agent that will attempt to determine if a user is computer literate based on user input. A user will be prompted with a series of computer related questions, and based off the answers, the conversational agent will conclude whether the user is computer literate or not. If time permits, the agent will be able to expand its vocabulary and learn new words.

SIGIFICANCE

An intelligent agent that comprehends emotion will be much easier for humans to interact with because they are more similar to us. After all, it’s easier to interact with a machine when you are not forced to think like a machine. As it becomes easier to interact with an intelligent agent, science will be able to find more practical applications of artificial intelligence. There are many people right now that are intimidated by technology because it is too difficult for them to interact with it. An artificial intelligence agent that is easy to work with will be a huge benefit to these sorts of people. They could comfortably interact with an emotional agent because it would understand their thoughts and feelings and react in a sympathetic manner.

Once an agent learns how to pick up on certain emotional cues, the agent will be able to be used in many different ways. If an intelligent agent is able to detect a user’s emotional state, it can change how it operates to better accommodate the user (Emotions in Human-Computer Interaction, 2003). For instance, imagine that a computer manufacturing company used our agent to conduct customer support. If the program initially detected that a customer had a low level of computer literacy, it could alter its execution so that it didn’t use potentially confusing computer rhetoric while helping the customer. On the other hand, if it detected that it was helping someone with a fairly strong understanding of computers, it could eliminate simple steps in the service process that the sophisticated customer might already know how to do.

In general, an emotionally competent agent will be a much better interface between humans and technology. If an agent can be developed to interact seamlessly and intuitively with a human, humans will be able to interact with computers the same way they would interact with another human. No extra skills would be necessary to take advantage of tremendous new technologies.

METHODOLOGY

Although our project isn’t as ambitious as some current emotional intelligence projects, we do hope to make interesting discoveries about how to extract emotional significance out of user input and reason about emotional aspects of the user. There are many different emotional conclusions that can be made based on a simple conversation with someone. For example, programs exist to determine the gender of a user based on text they’ve typed in. There are also other programs that ask a user a series of questions and then conclude if the user is of legal driving age. All of this is done by extracting the emotional content from the user’s responses and then analyzing it accordingly.

Our agent will have a slightly more limited domain. Our agent will be designed to pose a series of five questions to a user about computers. The questions will be broad enough to measure computer literacy based upon a fairly reasonable standard.

The first question asks what type of Internet browser the person uses the most. If the person is literate enough, they should be able to supply an accurate answer to this question. The second question pertains to the operating system the user runs. If they are fairly literate, they should be able to accurately name their specific operating system. The third question asks about what domain the person’s email address is under. This tests to see if a user knows what the word “domain” means with respect to a computer. The fourth question asks what computer languages the person knows. This is a more direct, explicit way of gauging their computer literacy. The final question asks what Internet search engine they use, as it seems that there is a relationship between a person’s computer savvyness and the search engine they favor.

Based on the user’s responses, the agent will determine if the user is computer literate. We will also enable the agent to make new associations between key terms so as to form primitive definitions of words it extracts from conversation. But the domain of these words will still be limited to computer literacy. For example, if the agent has already determined that the user is fairly literate, it will add new words it receives from the user to the Word library as words that indicate literacy. However, if the agent knows that it is talking to a fairly illiterate person, it will add their words to the library as flag words to indicate illiteracy. In this way, our library of Word objects will continually expand and allow for more precise designations of literacy levels based on empirical results.

The program will ultimately make a conclusion regarding the user’s level of computer literacy based on multiple factors. If a user doesn’t use much computer rhetoric (they don’t use words that match with many pre-defined computer terms), they will be judged to not be very computer literate. Or, if a user uses a word incorrectly in a sentence, they will be deemed illiterate. However, if a user enters responses that contain words that have been designated in the library as indicative of a literate person, that user will be deemed computer literate. Each word is assigned a value between zero and five, with zero being the lowest literacy and five being the highest. Whenever the user matches a word in the library, that word’s literacy value is added to an accumulator, and in the end the person’s literacy rating is that accumulated value divided by five times the total number of words that were matched, yielding a final literacy rating between zero and one. When adding new words, we look at the person’s literacy rating to assign a literacy value to a new word. If the person’s rating is between .8 and 1, the new word gets a value of four. If the rating is between .6 and .8, it gets a three. Between .4 and .6 it gets a two, and between .2 and .4 it receives a one. A zero is assigned if the person’s rating is zero, and a five is assigned if the person’s rating is a one. For simplicity’s sake, only words that are four characters long or bigger will be added, so as to eliminate a word library filled with words like “the”, “and”, or “it”.

Our test subjects will be a random pool of human test subjects that will interact with our agent. We will talk with the test subjects and get a feel for their computer literacy. This may seem inaccurate, but it is actually very germane to our project. Our agent’s main goal is to rate a person’s computer literacy by thinking about the user’s response like a human. So, to determine if our agent has done this successfully, the best thing to do is compare it’s evaluation to the evaluation of the user by an actual human. In essence, we will be judging our agent by comparing how it thinks to how we think and, therefore, analyzing just how similar it’s thought processes are to those of a human.

PROGRAM DESIGN

Our program has two main data structures: a Word object and a QuestionFrame object. A Word object holds the word as a global string variable and maintains a literacy rating for that word so if the word is used the literacy rating is awarded to the user. The Word object also contains a list of similar words in case a user doesn’t use the exact word, but rather uses a synonym. The Word objects are all placed in an array and compose a Word library that we utilize to rank people’s computer literacy. Word objects can have positive or negative literacy ratings. Positively rated Words are indicative of a computer literate person, negative Words are flag words that clearly demonstrate a person is illiterate.

The QuestionFrame object is a class that displays the GUI. It poses a question to the user and collects their response. These objects are recorded in an array so that each response can be extracted from each separate frame an analyzed at a later time.

A Setup class calls one QuestionFrame for each question the user will be asked. It waits until it gets a response from each one until another QuestionFrame pops up. Then, each response is parsed and each word in the response string is queried in the Word library. If it is found, the literacy rating assigned to it is added to the user’s overall literacy rating. If the word is not found, a new Word object is created for it. It is given a literacy rating based on the user’s current literacy rating via the process previously described.

EVALUATION

We determine the success of our program based on how accurately it identifies the computer literacy of a test user. As previously stated, we will have talked with the test subjects and determined their general level of literacy. Then we compare the result our agent gives us with the user’s original response. Garner said that he felt a 30% success rate in Turing Tests was progress. Since our scope is fairly limited, I feel we can be held to slightly higher standards. We hope that to achieve a 50% success rate with our program. That is, the intelligent agent should have been able to accurately determine the computer literacy of a person in half of all test cases. As the program learns (if we have had time to implement learning), the success rate will hopefully increase.

RESULTS

The results so far seem very close to what we expected. Based on a few words we have predefined in the library, we have been able to accurately gauge the literacy of a few test subjects. This may sound contrived, but as we test more people, we build our word library and therefore allow for more extensive testing.

CONCLUSIONS

Although we are very pleased with the results of our research, our agent could be significantly improved upon. Right now we use very primitive algorithms to match words and add words to our library. For instance, only words four characters or larger are added. A more sophisticated means of checking against superfluous words could be added. Our program is designed to be a minimal program, a starting point for starting deeper research on the topic. But now that we’ve developed this program, we can clearly see what would need to be changed to implement it on a larger scale.

Another aspect of our research that could be improved greatly is the series of questions we ask the user. The accuracy of our agent is largely contingent upon the quality of questions we pose to the user. One could make a completely separate research project out of discovering the factors that determine and define computer literacy. If more research was conducted upon what defines computer literacy, we could develop better questions that would allow us to determine a person’s computer literacy with much more precision.

In a world of constantly evolving technology, it is important that people feel comfortable and do not get left behind. Program machines and agents to be more receptive to human emotion is a necessary first step in allowing everyone to easily utilize technology. As we develop emotionally intelligent agents, we can finally force technology to adapt to us, rather than us having to adapt to it. By researching and developing agents that can extract emotional significance out of human words and sentences, we open up whole new frontiers in technological innovation.

BIBLIOGRAPHY

CMU Social Robots Project. Retrieved

September 10, 2003 from

Emotions in Human-Computer Interaction. Retrieved September 27, 2003 from

Generation5 – Robby Glen Garner. Retrieved September 26, 2003 from

Jurafsky, D. and James Martin. (2000). Speech and Language Processing. Prentice Hall. Retrieved from

Sociable Machines – Overview. Retrieved September 10, 2003 from

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download