Neuro-Symbolic Systems as Compression, Coordination, and Alignment

computer systems and Synthetic Intelligence, we had established establishments designed to cause systematically about human habits — the court docket. The authorized system is one in every of humanity’s oldest reasoning engines, the place information and proof are taken as enter, related legal guidelines are used as reasoning guidelines and verdicts are the system’s output. The legal guidelines, nevertheless, have been persistently evolving from the very starting of human civilization. The earliest Codified Regulation – the Code of Hammurabi (circa 1750 BCE) – represents one of many first large-scale makes an attempt to formalize ethical and social reasoning into express symbolic guidelines. Its magnificence lies in readability and uniformity — but it is usually inflexible, incapable of adaptation to context. Centuries later, Widespread Regulation traditions like these formed by the Case of Donoghue v Stevenson (1932), launched the other philosophy: reasoning based mostly on precedential expertise and circumstances. At present’s authorized programs, as we all know, are normally a mix of each, whereas the proportions differ throughout completely different international locations.

In distinction to the cohesive mixture in authorized programs, an identical paradigm pair in AI — Symbolism and Connectionism — appear to be considerably more durable to unite. The latter has dominated the surge of AI growth lately, the place every part is implicitly discovered with huge quantities of knowledge and computing assets and encoded throughout parameters in neural networks. And this route, certainly, has been confirmed very efficient when it comes to benchmark efficiency. So, do we actually want a symbolic part in our AI programs?

Symbolic Programs v.s. Neural Networks: A Perspective of Data Compression

To reply the query above, we have to take a better take a look at each programs. From a computational standpoint, each symbolic programs and neural networks will be seen as machines of compression — they cut back the huge complexity of the world into compact representations that allow reasoning, prediction, and management. But they accomplish that by basically completely different mechanisms, guided by reverse philosophies of what it means to “perceive”.

In essence, each paradigms will be imagined as filters utilized to uncooked actuality. Given enter (X), every learns or defines a change (H(cdot)) that yields a compressed illustration (Y = H(X)), preserving info that it considers significant and discarding the remainder. However the form of this filtering is completely different. Usually talking, symbolic programs behave like high-pass filters — they extract the sharp, rule-defining contours of the world whereas ignoring its easy gradients. Neural networks, against this, resemble low-pass filters, smoothing native fluctuations to seize international construction. The distinction is just not in what they see, however in what they select to overlook.

Symbolic programs compress by discretization. They carve the continual cloth of expertise into distinct classes, relations, and guidelines: a authorized code, a grammar or an ontology. Every image acts as a crisp boundary, a deal with for manipulation inside a pre-defined schema. The method resembles projecting a loud sign onto a set of human-designed foundation vectors — an area spanned by ideas corresponding to Entity and Relation. A data graph, for example, would possibly learn the sentence “UIUC is a rare college and I adore it”, and retain solely (UIUC, is_a, Establishment), discarding every part that falls exterior its schema. The result’s readability and composability, but additionally rigidity: that means exterior the ontological body merely evaporates.

Neural networks, in distinction, compress by smoothing. They forgo discrete classes in favor of easy manifolds the place close by inputs yield related activations (normally bounded by some Lipschitz fixed in trendy LLMs). Somewhat than mapping knowledge to predefined coordinates, they study a latent geometry that encodes correlations implicitly. The world, on this view, is just not a algorithm however a area of gradients. This makes neural representations remarkably adaptive: they will interpolate, analogize, and generalize throughout unseen examples. However the identical smoothness that grants flexibility additionally breeds opacity. Data is entangled, semantics grow to be distributed, and interpretability is misplaced within the very act of generalization.

In conclusion, we are able to summarize the distinction between the 2 programs from the knowledge compression perspective in a single sentence: “Neural Networks are blurry photographs of the world, whereas symbolic programs are high-resolution footage with lacking patches.” This really signifies the explanation why neuro-symbolic programs are an artwork of compromise: they will harness data from each paradigms through the use of them collaboratively at completely different scales, with neural networks offering a world, low-resolution spine and symbolic elements supplying high-resolution native particulars.

The Problem of Scalability

Although it is rather tempting so as to add symbolic elements into neural networks to harness advantages from each, scalability is a giant drawback getting in the way in which of our makes an attempt, particularly within the period of Basis Fashions. Conventional neuro-symbolic programs depend on a set of expert-defined ontology / schema / symbols, which is assumed to have the ability to cowl all potential enter circumstances. That is acceptable for domain-specific programs (for instance, a pizza order chatbot); nevertheless, you can not apply related approaches to open-domain programs, the place you will want consultants to assemble trillions of symbols and their relations.

A pure response is to go totally data-driven: as an alternative of asking people to handcraft an ontology, we let the mannequin induce its personal “symbols” from inner activations. Sparse autoencoders (SAEs) are a distinguished incarnation of this concept. By factorizing hidden states into a big set of sparse options, they seem to present us a dictionary of neural ideas: every characteristic fires on a specific sample, is (usually) human-interpretable, and behaves like a discrete unit that may be turned on or off. At first look, this seems to be like an ideal escape from the professional bottleneck: we not design the image set; we study it.

Right here (D) known as the dictionary matrix the place every column shops a semantically significant idea; the primary time period is the reconstruction loss of the hidden state (h), whereas the second is a sparsity penalty encouraging minimal activated neurons within the code.

Nevertheless, an SAE-only method runs into two elementary points. The primary is computational: utilizing SAEs as a dwell symbolic layer would require multiplying each hidden state by an unlimited dictionary matrix, paying a dense computation price even when the ensuing code is sparse. This makes them unimaginable for deployment at Basis Mannequin scales. The second is conceptual: SAE options are symbol-like representations, however they don’t seem to be a symbolic system — they lack an express formal language, compositional operators, and executable guidelines. They inform us what ideas exist within the mannequin’s latent area, however not methods to cause with them.

This doesn’t imply we should always abandon SAEs altogether — they supply substances, not a completed meal. Somewhat than asking SAEs to be the symbolic system, we are able to deal with them as a bridge between the mannequin’s inner idea area and the various symbolic artefacts we have already got: data graphs, ontologies, rule bases, taxonomies, the place reasoning can occur by definition. And a high-quality SAE skilled on a big mannequin’s hidden states then turns into a shared “idea coordinate system”: completely different symbolic programs can then be aligned inside this coordinate system by associating their symbols with the SAE options which can be persistently activated when these symbols are invoked in context.

Doing this has a number of benefits over merely inserting symbolic programs facet by facet and querying them independently. First, it allows image merging and aliasing throughout programs: if two symbols from completely different formalisms repeatedly mild up nearly the identical set of SAE options, we’ve robust proof that they correspond to the identical underlying neural idea, and will be linked and even unified. Second, it helps cross-system relation discovery: symbols which can be far aside in our hand-designed schemas however persistently shut in SAE area level to bridges we did not encode — new relations, abstractions, or mappings between domains. Third, SAE activations give us a model-centric notion of salience: symbols that by no means discover a clear counterpart within the neural idea area are candidates for pruning or refactoring, whereas robust SAE options with no matching image in any system spotlight blind spots shared by all of our present abstractions.

Crucially, this use of SAEs stays scalable. The costly SAE is skilled offline, and the symbolic programs themselves don’t must develop to “Basis Mannequin dimension” — they will stay as small or as massive as their respective duties require. At inference time, the neural community continues to do the heavy lifting in its steady latent area; the symbolic artefacts solely form, constrain, or audit behaviour on the factors the place express construction and accountability are Most worthy. SAEs assist by tying all these heterogeneous symbolic views again to a single discovered conceptual map of the mannequin, making it potential to match, merge, and enhance them with out ever setting up a monolithic, expert-designed symbolic twin.

When Can an SAE Function a Symbolic Bridge?

The image above quietly assumes that our SAE is “adequate” to function a significant coordinate system. What does that truly require? We don’t want perfection, nor do we want the SAE to outperform human symbolic programs on each axis. As an alternative, we want a couple of extra modest however essential properties:

– Semantic Continuity: Inputs that specific the identical underlying idea ought to induce related assist patterns within the sparse code: the identical subset of SAE options ought to are typically non-zero, reasonably than flickering on and off below small paraphrases or context shifts. In different phrases, semantic equivalence needs to be mirrored in a secure sample of lively ideas.

– Partial Interpretability: We wouldn’t have to grasp each characteristic, however a nontrivial fraction of them ought to admit sturdy human descriptions, in order that merging and debugging are potential on the idea stage.

– Behavioral Relevance: The options that the SAE discovers should really matter for the mannequin’s outputs: intervening on them, or conditioning on their presence, ought to change or predict the mannequin’s selections in systematic methods.

– Capability and Grounding: An SAE can solely refactor no matter construction already exists within the base mannequin; it can’t conjure wealthy ideas out of a weak spine. For the “idea coordinate system” image to make sense, the bottom mannequin itself needs to be massive and well-trained sufficient that its hidden states already encode a various, non-trivial set of abstractions. In the meantime, the SAE will need to have enough dimensionality and overcompleteness: if the code area is just too small, many distinct ideas might be compelled to share the identical options, resulting in entangled and unstable representations.

Now we talk about the primary three properties intimately.

Semantic Continuity

On the stage of pure operate approximation, a deep neural community with ReLU- or GELU-type activations implements a Lipschitz-continuous map: small perturbations within the enter can’t trigger arbitrarily unbounded jumps within the output logits. However this sort of continuity could be very completely different from what we want in a sparse autoencoder. For the bottom mannequin, a couple of neurons flipping on or off can simply be absorbed by downstream layers and redundancy; so long as the ultimate logits change easily, we’re happy.

In an SAE, against this, we’re not simply a easy output — we’re treating the assist sample of the sparse code reconstructed over the residual stream as a proto-symbolic object. A “idea” is recognized with a specific code subset being lively. That makes the geometry rather more brittle: if a small change within the underlying illustration pushes a pre-activation throughout the ReLU threshold within the SAE layer, a neuron within the code will out of the blue flip from off to on (or vice versa), and from the symbolic standpoint the idea has appeared or disappeared. There is no such thing as a downstream community to common this out; the code itself is the illustration we care about.

Sparsity penalty in setting up the SAE even exacerbates this. The standard SAE goal combines a reconstruction loss with an (ell_1) penalty on the activations, which explicitly encourages most neuron values to be as near zero as potential. Because of this, even many helpful neurons find yourself sitting close to the activation boundary: simply above zero when they’re wanted, just under zero when they don’t seem to be — this is named “activation shrinkage” in SAEs. That is unhealthy for semantic continuity on the assist sample stage: tiny perturbations within the enter can change non-zero neurons, even when the underlying that means has barely modified. Subsequently, Lipschitz continuity of the bottom mannequin doesn’t robotically give us a secure non-zero subset of code within the SAE area, and support-level stability needs to be handled as a separate design goal and evaluated explicitly.

Partial Interpretability

SAE defines an overcomplete dictionary to retailer potential options discovered from knowledge. Subsequently, we solely want a subset of those dictionary entries to be interpretable options. Even for that subset, meanings of the options are solely required to be roughly correct. After we align present symbols to the SAE area, it’s the activation patterns within the SAE layer that we depend upon: we probe the mannequin in contexts the place a logo is “in play”, report the ensuing sparse codes, and use the aggregated code as an embedding for that image. Symbols from completely different programs whose embeddings are shut will be linked or merged, even when we by no means assign human-readable semantics to each particular person characteristic.

Interpretable options then play a extra centered position: they supply human-facing anchors inside this activation geometry. If a specific characteristic has a fairly correct description, all symbols that load closely on it inherit a shared semantic trace (e.g. “these are all duty-of-care-like issues”), making it simpler to examine, debug, and arrange the merged symbolic area. In different phrases, we don’t want an ideal, totally named dictionary. We’d like (i) sufficient capability in order that vital ideas can get their very own instructions, and (ii) a sizeable, behaviorally related subset of options whose approximate meanings are secure sufficient to function anchors. The remainder of the overcomplete code can stay as nameless background; it nonetheless contributes to distances and clusters within the SAE area, even when we by no means title it.

Behavioral Relevance by way of Counterfactuals

A characteristic is simply fascinating, as a part of a bridge, if it really influences the mannequin’s habits — not simply if it correlates with a sample within the knowledge. In causal phrases, we care about whether or not the characteristic lies on a causal path within the community’s computation from enter to output: if we perturb the characteristic whereas holding every part else fastened, does the mannequin’s behaviour change in the way in which that its believed that means would predict?

Formally, altering a characteristic is much like an intervention of the shape (textual content{do}(z = c)) within the causal sense, the place we overwrite that inner variable and rerun the computation. However not like classical causal inference modeling, we don’t really want Pearl’s do-calculus to determine (P(y mid textual content{do}(z))). The neural community is a totally observable and intervenable system, so we are able to merely execute the intervention on the interior nodes and observe the brand new output. On this sense, neural networks give us the posh of performing idealized interventions which can be unimaginable in most real-world social or financial programs.

Intervening on SAE options is conceptually related however carried out in a different way. We sometimes have no idea the that means of an arbitrary worth within the characteristic area, so the exhausting intervention talked about above will not be significant. As an alternative, we amplify or suppress the magnitude of an present characteristic, which behaves extra like a smooth intervention: the structural graph is left untouched, however the characteristic’s efficient affect is modified. As a result of SAE reconstructs hidden activations as a linear mixture of a small variety of semantically significant options, we are able to change the coefficients of these options to implement significant, localized interventions with out affecting different options.

Symbolic-System Primarily based Compression as an Alignment Course of

Now let’s take a barely completely different view. Whereas neural networks compress the world into some extremely summary, steady manifolds, symbolic programs compress it right into a human-defined area with semantically significant axes alongside which the system’s behaviors will be judged. From this attitude, compressing info into the symbolic area is an alignment course of, the place a messy, high-dimensional world is projected onto an area whose coordinates mirror human ideas, pursuits, and values.

After we introduce symbols like “responsibility of care”, “menace of violence”, or “protected attribute” right into a symbolic system, we aren’t simply inventing labels. This compression course of does three issues directly:

– It selects which elements of the world the system is obliged to care about (and which it’s presupposed to ignore).

– It creates a shared vocabulary in order that completely different stakeholders can reliably level to “the identical factor” in disputes and audits.

– It turns these symbols into dedication factors: as soon as written down, they are often cited, challenged, and reinterpreted, however not quietly erased.

Against this, a purely neural compression lives totally contained in the mannequin. Its latent axes are unnamed, its geometry is personal, and its content material can drift as coaching knowledge or fine-tuning goals change. Such a illustration is superb for generalization, however poor as a locus of obligation. It’s exhausting to say, in that area alone, what the system owes to anybody, or which distinctions it’s presupposed to deal with as invariant. In different phrases, neural compression serves prediction, whereas symbolic compression serves alignment with a human normative body.

When you see symbolic programs as alignment maps reasonably than mere rule lists, the connection to accountability turns into direct. To say “the mannequin should not discriminate on protected attributes”, or “the mannequin should apply a duty-of-care normal”, is to insist that sure symbolic distinctions be mirrored, in a secure means, inside its inner idea area — and that we be capable of find, probe, and, if obligatory, appropriate these reflections. And this accountability is normally desired, even at the price of compromising a part of the mannequin functionality.

From Hidden Regulation to Shared Symbols

In Zuo Zhuan, the Jin statesman Shu-Xiang as soon as wrote to Zi-Chan of Zheng: “When punishment is unknown, deterrence turns into unfathomable.” For hundreds of years, the ruling class maintained order by secrecy, believing that worry thrived the place understanding ended. That’s why it grew to become a milestone in historic Chinese language historical past when Zi-Chan shattered that custom, forged the felony code onto bronze tripods and displayed it publicly in 536 BCE. Now AI programs are dealing with an identical drawback. Who would be the subsequent Zi-Chan?

References

• Bloom, J., Elhage, N., Nanda, N., Heimersheim, S., & Ngo, R. (2024). Scaling monosemanticity: Sparse autoencoders and language fashions. Anthropic.
• Garcez, A. d’Avila, Gori, M., Lamb, L. C., Serafini, L., Spranger, M., & Tran, S. N. (2019). Neural-symbolic computing: An efficient methodology for principled integration of machine studying and reasoning. FLAIRS Convention Proceedings, 32, 1–6.
• Gao, L., Dupré la Tour, T., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., & Wu, J. (2024). Scaling and evaluating sparse autoencoders.
• Bartlett, P. L., Foster, D. J., & Telgarsky, M. (2017). Spectrally-normalized margin bounds for neural networks. Advances in Neural Data Processing Programs, 30, 6241–6250.
• Chiang, T. (2023, February 9). ChatGPT is a blurry JPEG of the Net. The New Yorker.
• Pearl, J. (2009). Causality: Fashions, reasoning, and inference (2nd ed.). Cambridge College Press.
• Donoghue v Stevenson [1932] AC 562 (HL).