fashions related to your Neo4j graph achieve unbelievable flexibility: they will generate any Cypher queries by the Neo4j MCP Cypher server. This makes it attainable to dynamically generate complicated queries, discover database construction, and even chain multi-step agent workflows.
To generate significant queries, the LLM wants the graph schema as enter: the node labels, relationship sorts, and properties that outline the info mannequin. With this context, the mannequin can translate pure language into exact Cypher, uncover connections, and chain collectively multi-hop reasoning.
For instance, if it is aware of about (Particular person)-[:ACTED_IN]->(Film) and (Particular person)-[:DIRECTED]->(Film) patterns within the graph, it could actually flip “Which films function actors who additionally directed?” into a legitimate question. The schema offers it the grounding wanted to adapt to any graph and produce Cypher statements which can be each appropriate and related.
However this freedom comes at a price. When left unchecked, an LLM can produce Cypher that runs far longer than supposed, or returns monumental datasets with deeply nested buildings. The end result is not only wasted computation but in addition a critical threat of overwhelming the mannequin itself. For the time being, each device invocation returns its output again by the LLM’s context. Which means if you chain instruments collectively, the entire intermediate outcomes should circulate again by the mannequin. Returning 1000’s of rows or embedding-like values into that loop shortly turns into noise, bloating the context window and decreasing the standard of the reasoning that follows.
For this reason throttling responses issues. With out controls, the identical energy that makes the Neo4j MCP Cypher server so compelling additionally makes it fragile. By introducing timeouts, output sanitization, row limits, and token-aware truncation, we will maintain the system responsive and be certain that question outcomes keep helpful to the LLM as an alternative of drowning it in irrelevant element.
Disclaimer: I work at Neo4j, and this displays my exploration of potential future enhancements to the present implementation.
The server is out there on GitHub.
Managed outputs
So how will we stop runaway queries and outsized responses from overwhelming our LLM? The reply is to not restrict what sorts of Cypher an agent can write as the entire level of the Neo4j MCP server is to reveal the complete expressive energy of the graph. As an alternative, we place good constraints on how a lot comes again and how lengthy a question is allowed to run. In observe, meaning introducing three layers of safety: timeouts, end result sanitization, and token-aware truncation.
Question timeouts
The primary safeguard is easy: each question will get a time funds. If the LLM generates one thing costly, like an enormous Cartesian product or a traversal throughout hundreds of thousands of nodes, it’s going to fail quick as an alternative of hanging the entire workflow.
We expose this as an atmosphere variable, QUERY_TIMEOUT, which defaults to 10 seconds. Internally, queries are wrapped in neo4j.Question with the timeout utilized. This fashion, each reads and writes respect the identical certain. This variation alone makes the server way more strong.
Sanitizing noisy values
Trendy graphs usually connect embedding vectors to nodes and relationships. These vectors will be tons of and even 1000’s of floating-point numbers per entity. They’re important for similarity search, however when handed into an LLM context, they’re pure noise. The mannequin can’t purpose over them instantly, and so they eat an enormous quantity of tokens.
To resolve this, we recursively sanitize outcomes with a easy Python operate. Outsized lists are dropped, nested dicts are pruned, and solely values that match inside an inexpensive certain (by default, lists beneath 52 gadgets) are preserved.
Token-aware truncation
Lastly, even sanitized outcomes will be verbose. To ensure they’ll all the time match, we run them by a tokenizer and slice all the way down to a most of 2048 tokens, utilizing OpenAI’s tiktoken library.
encoding = tiktoken.encoding_for_model("gpt-4")
tokens = encoding.encode(payload)
payload = encoding.decode(tokens[:2048])This last step ensures compatibility with any LLM you join, no matter how massive the intermediate knowledge could be. It’s like a security internet that catches something the sooner layers didn’t filter to keep away from overwhelming the context.
YAML response format
Moreover, we will scale back the context measurement additional by utilizing YAML responses. For the time being, Neo4j Cypher MCP responses are returned as JSON, which introduce some further overhead. By changing these dictionaries to YAML, we will scale back the variety of tokens in our prompts, reducing prices and bettering latency.
yaml.dump(
response,
default_flow_style=False,
sort_keys=False,
width=float('inf'),
indent=1, # Compact however nonetheless structured
allow_unicode=True,
)Tying it collectively
With these layers mixed — timeouts, sanitization, and truncation — the Neo4j MCP Cypher server stays absolutely succesful however way more disciplined. The LLM can nonetheless try any question, however the responses are all the time bounded and context-friendly to an LLM. Utilizing YAML as response format additionally helps decrease the token depend.
As an alternative of flooding the mannequin with massive quantities of information, you come simply sufficient construction to maintain it good. And that, ultimately, is the distinction between a server that feels brittle and one which feels purpose-built for LLMs.
The code for the server is out there on GitHub.
