Within the Creator Highlight collection, TDS Editors chat with members of our neighborhood about their profession path in knowledge science and AI, their writing, and their sources of inspiration. At this time, we’re thrilled to share our dialog with Shuai Guo.
Shuai is an industrial AI researcher working with physics, knowledge, and machine studying to resolve real-world issues in engineering, safety, and clever methods. He holds a PhD on the intersection of computational mechanics and machine studying. His work spans varied matters, together with anomaly detection, digital twins, physics-informed studying, and LLM/agentic functions.
Your LangGraph piece walks the reader by means of the method of constructing a deep analysis agent. Whenever you really tried it end-to-end, what shocked you essentially the most, and what would you do otherwise subsequent time?
I might say what shocked me essentially the most was how simply the deep analysis agent could make errors when working it end-to-end. That complete “generate question → search → replicate → repeat” loop seems nice on paper, nevertheless it falls aside fairly quick. There are two primary points I bear in mind clearly. First, once in a while, the agent begins mixing up what it discovered with what it remembers from pre-training. This isn’t ultimate, as I solely need the LLMs to synthesize info and determine data gaps, whereas absolutely relying on the internet search to floor the reply.
One other concern that continuously provides me complications is info contamination, i.e. when search brings again related stuff however the mannequin treats it prefer it’s precisely what you requested for. For instance, I as soon as examined the deep analysis agent by researching a particular bug report (say, concern #4521 of a codebase), and the search would return content material associated to concern #4522 and begin mixing of their signs like they’re all the identical downside.
Past these two primary points, I additionally skilled challenges in dealing with conflicting info and figuring out sufficiency for terminating the deep analysis. None of these issues could be solved by merely including extra search outcomes or working extra iterations.
The important thing realization for me is that guardrails are as important, if no more so, than the agent structure, if we wish to transcend “only a demo” and construct a system that truly works in manufacturing. I feel the mindset of “test-driven improvement” matches properly right here: outline what “good” seems like earlier than you construct. Subsequent time, I’d begin by defining clear guidelines, after which construct the agent structure round these constraints.
You’ve written that analytical AI (SQL/BI + classical ML) isn’t going away just because agents are hot. Should you have been designing a contemporary knowledge stack right now, what work would you give to brokers and what would you retain within the analytics lane?
Analytical AI is reproducible and numerically exact. LLM-based brokers, however, are good at digesting unstructured context, translating outcomes, and speaking with folks. For allocating duties between analytical AI and agentic AI, I might say if a activity is extra quantitatively geared, I might default to analytical AI; but when it’s extra qualitatively geared, e.g., synthesis, storytelling, or judgment, I might think about LLM/brokers as higher options.
We will think about a concrete downside of constructing a buyer churn prediction system. On a excessive stage, it often includes two steps: figuring out the at-risk prospects, and appearing on them. For step one of flagging at-risk prospects, I might lean on analytical AI to engineer informative options, practice gradient boosting fashions on historic behavioral knowledge, and use the educated fashions to calculate churn propensity scores. As well as, I might additionally run a SHAP evaluation to get function significance scores for explaining the prediction. Each step is exact and reproducible, and there are a ton of finest practices out there for getting correct and dependable outcomes.
However then comes the enjoyable half: what do you really do with these predictions? That is the place the LLM-based brokers can take over. They’ll draft customized retention emails by pulling within the buyer’s historical past, possibly counsel related product options they haven’t tried but, and modify the tone primarily based on how their previous help tickets went. There is no such thing as a math right here. Simply talking in a contextually good approach.
What’s one ability you invested in early that now provides you a bonus as AI instruments get extra succesful?
Techniques pondering.
To me, methods pondering is mainly asking decompose methods into parts. How do totally different parts speak to one another? What are the handoff factors? The place are the suggestions loops? If I contact this, what else adjustments?
I picked this up at college. I majored in aerospace engineering with a give attention to aero-engine design. The factor about jet engines is that every thing impacts every thing, and finding out it actually helped me develop three habits: decompose the system, outline clear interfaces, and at all times look out for coupling results.
It’s true that AI instruments are getting extra succesful, e.g. we received higher coding assistants, more practical RAG pipelines, or LLMs that may deal with longer context, however a lot of the developments occur in slender slices. As an alternative of at all times chasing the most well liked device and making an attempt to include it one way or the other in my current work, methods pondering helps me put the massive image entrance and middle. For an LLM software, I might at all times begin by sketching the parts, figuring out interplay and inputs/outputs between the parts, ensuring checks and guardrails are added, after which swapping parts as instruments enhance.
The truth is, constructing LLM functions jogs my memory a variety of designing jet engines: new know-how comes and goes, however a stable system design compounds worth.
Should you zoom out, what a part of knowledge science or AI is altering too quick proper now, and what half isn’t altering quick sufficient?
I feel multi-agent AI methods are undoubtedly one of many hottest fields which can be transferring very quick. We see fancy demos (be it coding assistant or analysis assistant) now and again. New open-sourced frameworks that allow builders to effectively construct their very own multi-agent functions additionally pop up continuously. All of that is thrilling. However right here is the factor: are we pushing out these difficult methods approach quicker than we perceive how they’ll really behave in follow?
That’s the place I see the hole: the entire “assurance” layer round these multi-agent methods isn’t evolving quick sufficient. To deal with this problem, we will (and doubtless ought to) deal with these multi-agent methods similar to another industrial system. Within the manufacturing business, it’s a frequent follow to undertake data-driven approaches to help system design, management, situation monitoring, and fault evaluation. This identical method may benefit multi-agent methods as effectively. For example, how about we use Bayesian optimization to design the multi-agent structure? How about utilizing ML-based anomaly detection to observe the brokers’ efficiency and catch safety threats?
The excellent news is there’s momentum constructing. We’re seeing observability platforms for LLMs, analysis frameworks, and many others., and so they’re laying the groundwork for making use of these industrial-grade, data-driven strategies. I see a variety of alternatives on this house and that’s what will get me excited: the prospect to convey the rigor of business methods to agentic AI and make these instruments dependable and reliable.
To be taught extra about Shuai‘s work and keep up-to-date together with his newest articles, you may comply with him on TDS or LinkedIn.
