The New Natural Language Understanding



Machine Natural Language Understanding

Jed Leslie

Rensselaer Polytechnic Institute

Abstract

Natural language processing has been a problem for artificial intelligence researchers since the conception of the field in the mid 1900's. AI's most famous test of an agent's intelligence, the Turing Test, is fundamentally bound to the idea of natural language processing. Much work has been done in this sub field, and not without sufficient progress. But failure to address the most basic principals of natural language has hindered any significant breakthroughs. As I will disclose in this paper, our current methods are wrong, and will continue to fail unless they are changed. The grail of modern AI, building a machine which can pass the Turing Test, will remain unreachable unless we can accept AI NLP as a non logical field.

I. Introduction

A. Natural Language Defined

As you are reading this paper, you are experiencing the phenomenon that we call natural language. This is not an easy term to define, as it encompasses many realms of human intelligence and cognitive theory. Natural language (NL) can best be described as the conveying of ideas or states through a set of signs. These signs are generally communicated through the air (auditory) or as some sort of symbolic representation on something physical, as in the case of this paper or the characters on my computer screen. Natural language as we know it is a rather complex byproduct of the human desire to communicate knowledge about the world to each other. Although some advanced species show signs of a small set of language symbols (such as the dolphin and chimpanzee), most other species appear limited to the use of conventional signs, such as facial expressions, physical interactions with one another and bounded verbal signs (a dog's bark) [1]. Humans are the only species to have developed an unbounded vocabulary of language symbols which can be interpreted by other humans in an apparent infinite number of different contexts. English, Chinese and Spanish are all examples of natural language. Human natural language is so engrained into our lives that it is very difficult to appreciate the magnitude of its significance in the development of our lives as individuals and as a collective. But natural language isn't something that we are born with or something that is installed, it is something that is learned. Natural language is the dynamic product of human intelligence and our species' ever growing understanding of the universe we live in.

B. Human Natural Language

Studies have shown that humans have evolved something which enables us to learn, understand and communicate using natural language [2]. Noam Chomsky has done much work in the field of linguistics and the way it relates to human intelligence over the past half century. He was the first person to propose a theory of universal grammar (UG). The concept of UG is found in the very depths of the human brain and allows the assimilation and understanding of all human languages.

Chomsky's theory of universal grammar is the topic of much debate within the field of linguistics, but the significance of his suggestions should not be ignored in relation to our topic of AI and natural language. If Chomsky's theories are correct, getting to the root of UG should become a priority for NL researchers as it may provide the algorithms needed for machine natural language processing.

C. The Need for Machine NL

The potential benefits of a machine that can interact with humans using natural language are absolutely mind boggling. The need for advanced human-machine communication interfaces, such as the mouse and keyboard, would be rendered obsolete. The computing learning curve would almost completely disintegrate as machines would be able to instantly interpret human desires such as "open my payroll file," or "could you find me information on travel locations in New Zealand?" The entire design of the modern PC would change, from operating system to I/O. But personal computing is not the only field that could benefit from machine natural language. Science fiction has addressed the potentials very extensively, from the "Computer" in Star Trek to the speaking robot visions of Isaac Asimov as a tool for the advancement of human civilization [3]. Machine natural language could help us tutor and teach our youth, get help in times of emergency, have our coffee ready by morning and ultimately gain a greater understanding of the way our intelligence and consciousness work.

D. Historic Machine NL and the Turing Test

Since the 1950's, machine natural language processing has posed one of the greater challenges for artificial intelligence researchers. The Turing Test is Alan Turing's famous test of machine operational intelligence. Turing defined intelligence as the ability to achieve human-level performance in all cognitive tasks, sufficient to fool an interrogator [1]. In order to pass his test, a human interrogator would have to be unable to decipher if the test taker was a machine or a human. Despite 50 years of work and positive initial speculation, AI has failed (miserably) to produce an intelligent machine capable of passing the Turing Test. This is partly because the work needed to produce the four necessary capabilities that a machine would need to pass the test was drastically underestimated, and partly because of the ambiguity between actual intelligence and the operational intelligence that the test dictates.

The four necessary capabilities are knowledge representation, automated reasoning, machine learning, and natural language processing. Knowledge representation is the machine's ability to store information about the universe. Automated reasoning is the machine's ability to interpret the information and respond to different events based on that interpretation. Machine learning is the machine's ability to adapt it's knowledge representation and reasoning based on a changing environment. The forth and most important necessary capability in relation to this paper is natural language processing, or the machine's ability to communicate in some human language in such a manner as to sufficiently fool the interrogator.

Each necessary capability is in itself a whole field of artificial intelligence. While scattered progress has been made in each of these areas, a system which can truly bring together these fields has yet to emerge. I believe that the key to building a machine that can successfully pass the Turing Test lies in extensive work within the machine learning field. Knowledge representation, automated reasoning and natural language processing will all be derived from significant progress in machine learning.

E. Modern Machine NLP Methods

Today's efforts in machine natural language processing (NLP) are very mathematical. They involve extensive research into communicate theory, where an agent (or human) first has an intention to generate a speech phrase. It is then up to another agent (or human) to correctly perceive the speech phrase. The way humans perceive speech phrases and the way machines have been designed to perceive speech phrases appear drastically different.

Machines first must analyze the speech phrase and break up their analysis into different sentence parts. This process is called sentence parsing, where nouns, verbs, articles and adjectives are separated. The parsed elements are then interpreted using logical methods where semantic ambiguities are removed, a logical model is built and the machine's knowledge base is updated with the new information. For example, a machine model might interpret the phrase "Mary is sad" as NOT Happy(Mary, Now). Once updated, the machine's knowledge base will contain the information that Mary's current state is not happy. This may appear to be a good method for simple machine NLP tasks, but there is still no underlying understanding as to what "happy" is. The machine just can recognize that "happy" is the same as "not sad" and vice versa.

II. A Conceptual Failure

A. NL as a Human Construct

Natural language, unlike some other fields of artificial intelligence, must be addressed in a human like manner as it functions symbiotically with human intelligence. Humans do not parse language into it's associative word types (verb, noun, adjective, etc…) and attempt to build probabilistic models of meaning based on sentence structure. The paragraph is as important in deciphering the meaning of a particular sentence as the sentence itself. The physical setting or the state of the universe while the paragraph is being uttered is absolutely essential in correctly interpreting the meaning of the paragraph. Human natural language appears to be a dynamic symbolic representation of an individual's universe model. By this I suggest that before problems of machine natural language can be properly solved, machines will have to possess a universe model similar to that of a humans. This is not a new concept as it has been widely accepted since the beginning of AI that machines will need a form of knowledge representation in order to become operationally intelligent. It is within the realm of natural language processing that the idea of a human like universe model and the classic machine knowledge representation diverge. Being that natural language is a human construct, which evolved from the social need to convey ideas about the universe based on human perception, machines can only acquire natural language understanding through the harmonious and simultaneous acquisition of a universe model through human like perception and social influence.

B. The Universe Model Necessity

The symbolic translation of natural language is available to humans because of our extensive universe models. Natural language should not be viewed as a layer above human intelligence, but more as in integrated component. A universe model is made up of the way we perceive the world around us, the forces of the universe, the physical constraints of our own bodies, and the social interaction with others.

Our senses of sight, smell, taste, sound, touch, balance and spatial relations (such as the ability to touch your nose with your eyes closed) are constantly providing our brains with raw information about the universe. These senses have been shown to be intricately bound to one another, enabling a dense set of relational sensory information

[7]. The combination of these senses and the brain's ability to filter out unnecessary information (allowing focus on a particular event or object in the universe) build the first perceptual ideas of a universe model. Understanding the way something will feel in relation to the way it looks, or the way something will taste in relation to the way it smells are all bits of knowledge acquired through sensory input. It is important to note that not all information from the senses is believed to be stored in the brain. Much of the information remains in the universe as humans only model the information that is immediately relevant [8]. A machine that can not model the universe using human like senses will not have any understanding of descriptive phrases like "it looks like…" or "it smells like…"

The forces of the universe along with the evolved physical constraints of our own bodies play a huge role in the development of our universe models and ultimately our intelligence. This could be one of the most overlooked aspects of the origins of human intelligence in AI. Much of what we learn about movement and control over our bodies stems from what we can physically do within the universe. From the infant stages, gravity, friction and kinetic forces help build our universe models as we learn how to move our muscles and joints to either utilize, remain neutral or contradict these forces. A machine who can not physically interact with the universe, and does not share similar physical constraints (degrees of freedom), will not be able to understand even the most basic elements of human natural language.

Above all, social interaction plays the most important part in helping the developing human build an understanding of natural language. Just as the sense of sight can associate red with a fire truck, natural language understanding allows the infant to associate the way the word "red" sounds with the way a fire truck is perceived. This association is dependent on the interaction of the developing human and its caregiver. Social settings also help build universe models though what is called learning through scaffolding [7]. Scaffolding may include reducing distractions (gaining focus), marking the task's critical elements, or physical assistance (when a mother helps a baby use a spoon by holding the baby's hand and guiding it to the baby's mouth).

I believe that most of the universe model in relation to natural language is built through social interaction. When a mother says to a child "look at the beautiful flower" the child's universe model is altered to understand what a flower looks like, smells like and that those looks and smells are beautiful.

A universe model is not something that can be installed. The internal representations of a universe model are unimportant, as they will differ significantly from person to person (perhaps this is the definition of somebody's taste or preference). It's the acquisition of information about the universe through all the shared human elements simultaneously that make us able to use natural language so extensively.

Above all, the building of a universe model is an incremental process. We can not learn to walk until we learn to crawl. We can not learn to crawl until we learn to sit up. We can not read Romeo and Juliet until we read See spot run.

C. The Illogical Machine Method

So with all this evidence from cognitive psychology and neuroscience about the development of human intelligence, why do we continue to use logical methods in our quest to build machines which can communicate with us using natural language?

First, I would like to suggest that the idea of natural language as an evolved element of human intelligence has not been properly addressed. Evolution shows us that as species approach higher intelligence states, as in the case of dolphins, chimpanzees and humans, they appear to develop natural language as a tool for survival.

Second, diving deeper into the depths of cognitive theory, I would like to suggest that high intelligence is enabled by the ability to convey information about the universe to others through natural language, not the opposite. As the old saying goes, two heads are better then one. Sharing information has allowed the human to climb to the very top of the evolutionary chain. We have evolved with something in our brains that suggests that language is an inherent property of modern human existence. The actual specifics of the language, such as the semantics of English or Chinese are irrelevant, as young children will learn any language that their environment exposes them to with equal fluidity [12]. A striking example of this is the fact that many twins develop a personal language between themselves at a very young age. This language sounds like gibberish to all but each other. Most twins spend nearly all their young lives together constantly learning from one another. The twin example highlights much of the social interaction theory as the key to the development of intelligence in a young human.

Current methods in the development of machine natural language will continue to fail as the fundamental principal of natural language being a fully integrated part of a universe model remains unrecognized.

III. Understanding NL through Universe Exposure

A. The Human Infant

At birth, the body of the human infant is brought into the world 100% dependant on the care of others. If not properly nourished, the baby will die. The brain of the human infant is equally dependent on nourishment for it's development. Even the most basic perceptual senses are not developed until after birth [7]. As time progresses, the infant learns about it's environment guided by a number of key elements. Through physical constraints, such as those posed by its own body's range of movement and universal forces such as gravity and pressure, the infant begins to build a universe model. But the physicality of the universe can only go so far in the progression of the infant's growing intelligence. Social interaction is the basis for the development of an infant's mind. Interaction with it's caregiver provides fundamental ideas of social expectations and roles. As the baby learns about its universe and its ability to interact with it, the social interaction develops the mind. It is during the extremely critical first few years of existence that the infant develops the ability to communicate its desires through speech [7]. Human infants have an evolutionary ability to understand natural language, but that ability is nothing more then a potential. Speech comes first, the ability to read natural language in it's printed form comes second. How does a child learn to read? By having somebody read to them. This builds the association between auditory symbols and the printed symbol. It is through the child's universe model that either can evoke particular meaning.

The point being is that natural language evolves in complete conjunction with the infant's physical, social and intelligence development. Expecting a machine to possess human like natural language understanding is completely impossible without proper and similar evolution of physical and social components.

B. The Machine Infant and the COG Project

Much of what I suggest as a guide for machine natural language understanding has already been attempted, but with a more general purpose, by Rodney Brooks and his team at MIT. Brooks was one of the first people in AI to truly look into neuroscience evidence as a tool for building machines with human-like intelligence [11]. The COG project marks significant progress within this newly forming area of AI [7]. Brooks' team built a robot modeled after a human torso (two arms, one head and similar degrees of freedom). The robot has cameras for eyes, microphones for ears and a set of gyroscopes to give it a sense of balance. The robot was not programmed to do any particular task, but rather was given the potential to learn how to perform any task. Much like a human infant, the robot knew nearly nothing about how to control it's "body" physically in the universe. The robot was put through several unsupervised reaching exercises, as an example of it's learning capabilities. The robot was instructed to reach towards a target in front of it with no implicit instructions on how to move it's arm (besides the ones defined by the universe and the physical constraints of it's motor abilities). The first few reach attempts where horribly off, but after a few hours of attempts, the robot was able to accurately reach towards the target [7].

This is the type of adaptive methodology that will push machine intelligence into the future areas of AI. Human-like intelligence can only be acquired through human-like interaction with the universe. While this may not be the goal for all areas of AI, it is certainly extremely relevant to natural language understanding.

IV. Machine Natural Language Understanding

A. Defining Initial Conditions

Building a machine that can interact with the universe as I have described will not be an easy task. There are clearly many elements that need to be solidly thought out before any work within this field can be attempted. However, I believe a project even of this magnitude will be far simpler then any logical efforts to define what knowledge, the universe and languge is in a machine. I believe that the key lies in allowing the machine to define its own universe model. Higher levels of intelligence will follow much like the human infant.

It is important to note that human intelligence has grown out of survival needs. Inherent properties, such as the need for food, rest and reproduction are not be ignored in the quest to build a machine with human-like intelligence. These properties define how we act in many situations. They can be considered a set of human desires.

Our intelligent machine must share these desires if it is to understand human natural language. I recognize that defining these initial conditions may be the most difficult part of the project. Theory of mind ideas like these have been discussed by Brooks' team, but more work will be needed in order truly understand how to implement these desires in a machine [9].

The intelligent machine will also need to have certain reactions which we commonly call instincts. Looking towards a loud noise, or feeling the need to protect the physical structure of one's body (feeling pain) are both examples of necessary instincts.

With proper initial conditions, based on the built-in survival needs of human beings, the first steps towards machine natural language understanding will be in place.

B. Simple Beginnings

Much like the human infant, the intelligent machine will need to enter the world in a state of near helplessness. Through its senses, the physicality of the universe and most importantly, the social interaction with humans, it will begin to build a universe model which will enable it's future understanding of natural language.

The intelligent machine will also need to spend most of it's time "listening" to the interactions of people. Like a human infant, it would be expected to only understand and say simple things at first. As it's universe model grows will it be able to form natural language on the level of human beings.

Starting simple should involve bootstrapping techniques to allow the machine to build on the knowledge it already has about the universe. For example, have a human "show" the machine an object, describe the object (or even just name the object), and the test the machine's ability to recognize the object. Techniques like this exploit what we already know about the way humans learn through repetition and trail and error.

C. Technology

Theory aside, it is important to address how these ideas may be accomplished. There has been great progress within the computational intelligence field of AI. This field focuses on three practical methods of implementing machines that can "learn" from their environment: artificial neural networks, genetic algorithms and fuzzy logic. Of these three fields, I believe the correct implementation of an artificial neural network is the most promising.

The basic building blocks for artificial neural networks (ANNs), called neurons, are modeled after the way the cells in the human brain are believed to store and process information. Studies have shown that information is stored in a distributed manner throughout the brain [5]. Learning is achieved through constant trials and feedback. First introduced by Warren McCulloch and Walter Pitts in 1943, the artificial neuron, shown in figure 4.2 bears strong resemblance to the biological neuron shown in figure 4.1. Both act on the principal of weighted inputs from other neurons which are then transformed using a dynamic function into outputs.

[pic]

Figure 4.1

[pic]

Figure 4.2

Even the fastest modern computer lacks the ability to perform pattern recognition tasks at even the most basic level. Logic and mathematics have been the framework for the software that runs on these machines. But where conventional methods fail, ANNs have been able to (at least partially) pick up the slack. ANNs are pros at pattern recognition and retrieval. From alarm clocks that take voice commands to cars that adapt to your driving habits, the possibilities for ANN applications seem endless.

The diverse fields and applications of language processing and character recognition have the most to gain from the use of ANNs. Tools such as text-to-speech conversion, auditory input for machines, automatic language translation, secure voice keyed locks, automatic transcription, aids for the deaf and for the physically disabled which respond to voice commands, and (my favorite) natural language processing, recognition and response are all flogged with ANN potential. The future will only bring better and more accurate uses of these technologies.

An ANN's ability to recognize dynamic and changing patterns will bring life to ideas previously only heard about in science fiction. Automobiles that drive themselves based on changing scenery and what the road looks like are quickly becoming a reality. Pattern recognition systems implemented with ANNs can figure out if there is a bomb in somebody's airplane luggage and may be able to pick out fugitive terrorists using facial analysis.

Lastly, ANNs can play a enormous roll in the progression of human health and wellness. The pattern recognition and analysis abilities of ANNs make it a valuable tool for the interpretation and decoding of the human genome. When the genome is understood, and all genes are identified, genetic medicines can be costumed tailored for each patient's personal needs. Near perfect medicine is an obvious benefit to all of humankind and all of our futures.

Most of today's ANNs capable of the rather advanced computational tasks that I have described use only two or three layers of about thirty or less neurons. These networks are usually implemented in software in a two dimensional manner. Contrary to these "simple" ANNs, the human brain is a three dimensional network of nearly 100 billion neurons [1]. The computational power of the human brain is clearly superior to even the most advanced ANN. Modern artificial neural networks will not be able to perform the advanced modeling techniques required for the implementation of machines with natural language understanding, but the ideas of ANNs are fundamentally sound and have proven their ability to learn from a changing environment.

I believe that future advances in ANNs and other learning technologies will enable this vision of a machine with natural language understanding. Through the careful coupling of perception, physical forces and constraints, universe modeling and social interaction, the ANNs of the future could provide human-like world interaction to a machine.

D. Potentially Interesting Experiments

The success of this project could lead to the development of many interesting experiments within the realms of AI, cognitive psychology and neuroscience. There are several scenarios which immediately come to mind.

First, design an intelligent machine which possess no initial conditions, or desires which are drastically different from those defined by human survival needs. The interesting element of this experiment would be whether or not the machine becomes intelligent at all. Without similar survival drives, the machine may decide that it is content in its "dumb" state. The machine may also evolve into an intelligence with a universe model drastically different then anything human like. This could provide us with invaluable insight into our own minds.

A second experiment may flow along the lines of the twin example. Design two machines to interact with each other, along with a human caretaker. Perhaps a similar development of a gibberish language will take place.

Giving a machine the ability to perceive the universe in ways vastly superior to humans might also yield an interesting experiment. Why limit it's vision to that of natural light when we can design instruments to pick up frequencies all across the spectrum? Why limit it's auditory sensitivity range to between twenty and twenty thousand hertz? Providing a machine with "super senses" may hinder it's ability to view the world in a human like fashion, but would give us dramatic insight into the minds of other creatures.

V. Intelligence vs. Consciousness

A. Necessary Separation

Throughout this paper I have spoken a lot about "intelligence." As all AI researchers know, this is an extremely hard term to define. I must point out that it is extremely important to separate intelligence, as I have referred to it in this paper, from consciousness. Although I do not rule out the possibility, it is hard to convince myself that, even if successful at communicating with humans through fluid natural language, our intelligent machine would be a conscious machine. This also is highly dependent on how one chooses to define consciousness.

A being is generally thought to be conscious if it recognizes that it itself (or some part of it) exists in some way. I don't think I could have produced a more vague definition, or for that matter, a better one. The point being is that consciousness is an extremely relative term.

B. Consciousness as a Product of Increasing Intelligence

I believe that consciousness as we know it is something which evolves out of growing intelligence. None of us can remember being born, only some vague point in time after. I would like to suggest that as the human infant learns about its environment, the physics of the universe, the constraints of it's own body, the social interactions with others and the methods of which to convey information to others through natural language, it comes into consciousness. We all must accept that there was some point in each of our lives where we were interacting on an intelligent level with others, but were not aware of it. The universe model presents self analyzation, which builds almost a recursive model of thought. Humans are constantly thinking to themselves in natural language. They are constantly running over mental scenarios to help them choose the correct way to interact with the universe. This is a type of thought that comes from vast experience, and ultimately the superior hardware of the human brain. This recursive model of thought could be the very element that defines consciousness.

This is certainly an abstract topic, but it is by no means invalid in assessing the potential of a successfully intelligent machine. If consciousness is derived from a heightened awareness of one's environment, a strong universe model, and the ability to self analyze, there is no reason to doubt that our intelligent machine will gain some form of consciousness given enough time. Whether or not it corresponds to the human-like consciousness we all feel inside of us will be as interesting as the quest to build this machine itself.

VI. The Unexplained

A. Humor

In as much liberty I have taken in describing my theories on human intelligence, natural language and consciousness, there are at least two elements of humanity which I can only question.

As Commander Data from Star Trek knows, humor is not an easy concept to understand. Why do humans find things funny? What makes something funny to one person and not to another? The only suggestion I have is that humor somehow evokes a discrepancy in an individual's universe model, or exposes some previously unknown element of the universe model of another. Lots of situational comedy is funny because it exposes ideas about the universe in which we don't normally take as valid. But how do humans come up with these ideas? Through the concept we call imagination.

B. Dream States

Dreams are another element of humanity which are extremely hard to describe. The concepts and events that happen in human dreams often could never physically happen in the universe, and are usually very reflective of the feelings, ideas and emotions that the dreaming person has been experiencing in recent memory.

Perhaps a dream is what an active human brain does when the majority of its sensory input is shut off or limited by the biological need for sleep. Dreams could be the recursive analysis of one's own thoughts with less of the physical constraints of the universe coming into play.

References

[1] Artificial Intelligence A Modern Approach

Stuart Russell, Peter Norvig

Copyright 1995 Prentice-Hall Inc.

[2] Universal Grammar and Linguistics

Michael Albert



[3] Robot Dreams

Isaac Asimov

Copyright 1950 Isaac Asimov

[4] The Self Organizing Map

Teuvo Kohonon

1990 IEEE Invited Paper

[5] Neuro-Fuzzy and Soft Computing

J.-S. R. Jang, C.-T. Sun, E. Mizutani

Copyright 1997 Prentice-Hall Inc.

[6] Recurrent Neural Networks for Prediction

Danilo P. Mandic, Jonathon A. Chambers

Copyright 2001 John Wiley & Sons, Ltd

[7] The COG Project: Building a Humanoid Robot

Rodney A. Brooks, Cynthia Breazeal, Matthew Marjanovic, Brian Scassellati, Matthew M. Williamson

[8] Memory Representations in natural tasks

D. Ballard, M. Hayhoe, J. Pelz

1995 Journal of Cognitive Neuroscience

[9] Humanoid Robots: A New Kind of Tool

Rodney A. Brooks, Cynthia Breazeal, Bryan Adams, Brian Scassellati

[10] The Relationship Between Matter and Life

Rodney A. Brooks

Copyright 2001 McMillan Magazines Ltd

[11] Intelligence Without Representation

Rodney A. Brooks

MIT Artificial Intelligence Lab 1987

[12] Assessing Language Development in Bilingual Preschool Children

Barry McLaughlin, Antoinette Gesi Blanchard, Yuka Osanai



................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download