What Is Neuro-Symbolic AI And Why Are Researchers Gushing Over It?

By creating a more human-like thinking machine, organizations will be able to democratize the technology across the workforce so it can be applied to the real-world situations we face every day. This creates a crucial turning point for the enterprise, says Analytics Week’s Jelani Harper. Data fabric developers like Stardog are working to combine both logical and statistical AI to analyze categorical data; that is, data that has been categorized in order of importance to the enterprise. Symbolic AI plays the crucial role of interpreting the rules governing this data and making a reasoned determination of its accuracy. Ultimately this will allow organizations to apply multiple forms of AI to solve virtually any and all situations it faces in the digital realm – essentially using one AI to overcome the deficiencies of another.

Neuro-symbolic artificial intelligence is a novel area of AI research which seeks to combine traditional rules-based AI approaches with modern deep learning techniques. Neuro-symbolic models have already demonstrated the capability to outperform state-of-the-art deep learning models in domains such as image and video reasoning. They have also been shown to obtain high accuracy with significantly less training data than traditional models.

Symbolic AI: The key to the thinking machine

Summarizing, neuro-symbolic artificial intelligence is an emerging subfield of AI that promises to favorably combine knowledge representation and deep learning in order to improve deep learning and to explain outputs of deep-learning-based systems. Neuro-symbolic approaches carry the promise that they will be useful for addressing complex AI problems that cannot be solved by purely symbolic or neural means. We have laid out some of the most important currently investigated research directions, and provided literature pointers suitable as entry points to an in-depth study of the current state of the art. We investigate an unconventional direction of research that aims at converting neural networks, a class of distributed, connectionist, sub-symbolic models into a symbolic level with the ultimate goal of achieving AI interpretability and safety. To that end, we propose Object-Oriented Deep Learning, a novel computational paradigm of deep learning that adopts interpretable “objects/symbols” as a basic representational atom instead of N-dimensional tensors (as in traditional “feature-oriented” deep learning).

The output of a classifier (let’s say we’re dealing with an image recognition algorithm that tells us whether we’re looking at a pedestrian, a stop sign, a traffic lane line or a moving semi-truck), can trigger business logic that reacts to each classification. Contemporary deep learning models are limited in their ability to interpret while the requirement of huge amounts of data for learning goes on increasing. Due to these limitations, researchers are trying to look for new avenues by uniting symbolic artificial intelligence techniques and neural networks. We introduce the Deep Symbolic Network model, which aims at becoming the white-box version of Deep Neural Networks . The DSN model provides a simple, universal yet powerful structure, similar to DNN, to represent any knowledge of the world, which is transparent to humans.

Deep learning and neuro-symbolic AI 2011–now

A little built-in symbolism can go a long way toward making learning more efficient; LeCun’s own success with a convolution (a built-in constraint on how neural networks are wired) makes this case nicely. AlphaFold 2’s power, which derives in part from carefully constructed, innate representations for molecular biology, is another. A brand-new paper from DeepMind showing some progress on physical reasoning in a system that builds in some innate knowledge about objects is a third. Rather, as we all realize, the whole game is to discover the right way of building hybrids. In order to tackle these types of problems, the researchers looked for a more data-driven approach and because of the same reason, the popularity of neural networks reached its peak.

The agenda is a balance of educational content on neuro-symbolic AI and a discussion of recent results. We’re developing technological solutions to assist subject matter experts with their scientific workflows by enabling the Human-AI co-creation process. Developing a general knowledge representation framework to facilitate effective reasoning over multiple sources of imprecise knowledge. Forward chaining inference engines are the most common, and are seen in CLIPS and OPS5.

IBM, MIT and Harvard release “Common Sense AI” dataset at ICML 2021

Horn clause logic is more restricted than first-order logic and is used in logic programming languages such as Prolog. Extensions to first-order logic include temporal logic, to handle time; epistemic logic, to reason about agent knowledge; modal logic, to handle possibility and necessity; and probabilistic logics to handle logic and probability together. Others, like Frank Rosenblatt in the 1950s and David Rumelhart and Jay McClelland in the 1980s, presented neural networks as an alternative to symbol manipulation; Geoffrey Hinton, too, has generally argued for this position. You will require a huge amount of data in order to train modern artificial intelligence systems. While the human brain has the capacity to learn using a limited number of examples, artificial intelligence engineers need to feed huge amounts of data into an artificial intelligence algorithm.

When faced with a new problem, CBR retrieves the most similar previous case and adapts it to the specifics of the current problem. E.g., John Anderson provided a cognitive model of human learning where skill practice results in a compilation of rules from a declarative format to a procedural format with his ACT-R cognitive architecture. For example, a student might learn to apply “Supplementary angles are two angles whose measures sum 180 degrees” as several different procedural rules. E.g., one rule might say that if X and Y are supplementary and you know X, then Y will be X.

Differences between Inbenta Symbolic AI and machine learning

Deep learning and neural networks excel at exactly the tasks that symbolic AI struggles with. They have created a revolution in computer vision applications such as facial recognition and cancer detection. A key component of the system architecture for all expert systems is the knowledge base, which stores facts and rules for problem-solving.The simplest approach for an expert system knowledge base is simply a collection or network of production rules. Production rules connect symbols in a relationship similar to an If-Then statement.

Extend the scope of search methods from gradient descent to graduate descent, allowing the exploration of non-differentiable solution spaces, in particular solutions expressed as programs. I would argue that the crucial part here is not the “gradient”, but it is the “descent”, or the recognition that you need to move by small increments around your current position (also called “graduate descent”). If you do not have a gradient at your disposal, you can still probe for nearby solutions and figure out where to go next in order to improve the current situation by taking the best among the probed locations. Having a gradient is simply more efficient , while picking a set of random directions to probe the local landscape and then pick the best bet is the least efficient.

A gentle introduction to model-free and model-based reinforcement learning

It had the first self-hosting compiler, meaning that the compiler itself was originally written in LISP and then ran interpretively to compile the compiler code. An example is the Neural Theorem Prover, which constructs a neural network from an AND-OR proof tree generated from knowledge base rules and terms. Tom Mitchell introduced version space learning which describes learning as search through a space of hypotheses, with upper, more general, and lower, more specific, boundaries encompassing all viable hypotheses consistent with the examples seen so far. More formally, Valiant introduced Probably Approximately Correct Learning , a framework for the mathematical analysis of machine learning. In contrast to the knowledge-intensive approach of Meta-DENDRAL, Ross Quinlan invented a domain-independent approach to statistical classification, decision tree learning, starting first with ID3 and then later extending its capabilities to C4.5.

If such an approach is to be successful in producing human-like intelligence then it is necessary to translate often implicit or procedural knowledge possessed by humans into an explicit form using symbols and rules for their manipulation. Artificial systems mimicking human expertise such as Expert Systems are emerging in a variety of fields that constitute narrow but deep knowledge domains. Parsing, tokenizing, spelling correction, part-of-speech tagging, noun and verb phrase chunking are all aspects of natural language processing long handled by symbolic AI, but since improved by deep learning approaches. In symbolic AI, discourse representation theory and first-order logic have been used to represent sentence meanings. Latent semantic analysis and explicit semantic analysis also provided vector representations of documents. In the latter case, vector components are interpretable as concepts named by Wikipedia articles.

The expert system processes the rules to make deductions and to determine what additional information it needs, i.e. what questions to ask, using human-readable symbols. For example, OPS5, CLIPS and their successors Jess and Drools operate in this fashion. Herbert Simon and Allen Newell studied human problem-solving skills and attempted to formalize them, and their work laid the foundations of the field of artificial intelligence, as well as cognitive science, operations research and management science. Their research team used the results of psychological experiments to develop programs that simulated the techniques that people used to solve problems. This tradition, centered at Carnegie Mellon University would eventually culminate in the development of the Soar architecture in the middle 1980s.

(One of the earliest papers in the field, “A Logical Calculus of the Ideas Immanent in Nervous Activity,” written by Warren S. McCulloch & Walter Pitts in 1943, explicitly recognizes this possibility). Nervous systems – Cyberneticists were particularly interested in human and animal nervous systems. They saw these as the key to intelligence, but were not dogmatic about how the principles of nervous systems could be replicated in actual machines. It follows that neuro-symbolic AI combines neural/sub-symbolic methods with knowledge/symbolic methods to improve scalability, efficiency, and explainability.

• The symbolic artificial intelligence is entirely based on rules, requiring the straightforward installation of behavioral aspects and human knowledge into computer programs.
• Many of the concepts and tools you find in computer science are the results of these efforts.
• If machine learning can appear as a revolutionary approach at first, its lack of transparency and a large amount of data that is required in order for the system to learn are its two main flaws.
• Good macro-operators simplify problem-solving by allowing problems to be solved at a more abstract level.
• More advanced knowledge-based systems, such as Soar can also perform meta-level reasoning, that is reasoning about their own reasoning in terms of deciding how to solve problems and monitoring the success of problem-solving strategies.
• They tended to focus on characteristics that both humans and animals had in common, such as activity and purposeful behaviour.

It is rather the requirement for end-to-end differentisymbolic ai of the architecture, so that some form of gradient-based method can be applied to learning. It is a very strong constraint applied to the type of solutions that are explored and is presented as the only option if you don’t want to do an exhaustive search of the solution space, which obviously would not scale . And unlike symbolic AI, neural networks have no notion of symbols and hierarchical representation of knowledge. This limitation makes it very hard to apply neural networks to tasks that require logic and reasoning, such as science and high-school math.

• Instead, they produce task-specific vectors where the meaning of the vector components is opaque.
• It also provides deep learning modules that are potentially faster and more robust to data imperfections than their symbolic counterparts.
• Moreover, they lack the ability to reason on an abstract level, which makes it difficult to implement high-level cognitive functions such as transfer learning, analogical reasoning, and hypothesis-based reasoning.
• In contrast, a multi-agent system consists of multiple agents that communicate amongst themselves with some inter-agent communication language such as Knowledge Query and Manipulation Language .
• This is true in supervised learning, but also in unsupervised learning, where large datasets of images or videos are assembled to train the system.
• With more linguistic stimuli received in the course of psychological development, children then adopt specific syntactic rules that conform to Universal grammar.