Graph RAG: Building Smarter Retrieval Workflows with Knowledge Graphs

Transcript

Cassie Shum: I'm going to talk about GraphRAG and building smarter retrieval workflows with knowledge graphs. There's a lot of words in there and I'll talk through what all of that actually means. First, just a little bit about myself, so you know the context. We're going to be talking about context a lot, but my background. I worked at Thoughtworks for quite some time. A lot of my experience is around the enterprise and multiple customers and big systems. Throughout my talk, with that context, that's actually what I have in my head is like, how do we scale to big enterprise? How do we utilize some of these tools in order to make that useful for them? You'll see a bit of a demo I'm doing on like toy demo, but if now we want to think about that at a large scale. That's a little bit of my background there.

I moved actually to a startup called RelationalAI. I'll be talking about knowledge graphs because that is actually what we care about and what we love. A lot about the knowledge graphs that I'll talk about is like, how do you actually set context for these AI and agent systems and what GraphRAG actually means here? For the past couple years, I led field engineering, which meant I still cared very much about the customers and the enterprise. I was in the buildings with the enterprises and trying to build relational knowledge graphs for them. I'm realizing that there is a basic understanding of what knowledge graph is. I decided to move into product engineering and lead the ecosystem team, which means, how do we become very developer centric? How do we actually say, how do we build these knowledge graphs from scratch? There's a little bit of that context of why I'm talking about these kinds of things today.

Objectives

The rule of threes, what are we going to cover today? I want to kick us off into where traditional RAG breaks down. I'll get into this a little bit, but when I came up with this abstract, it was a while ago. I was like, yes, GraphRAG is all the rage and the traditional RAG will not do what it can do. Of course, as I was not last-minute building my deck, I was like, I don't know how powerful GraphRAG actually is anymore, given how powerful these models and LLMs have progressed over the last six months. I love Hilary's talk around, let's stay adaptable. I had to adapt the talk. I think the way we actually think about our product is we're adapting and taking in all the information of how things are adapting, the models are changing, and what does that actually mean for us? Some of the things that I will take away with you guys is fundamentals and foundations.

I'm an architect at heart and building complex systems take componentization and good architecture and good design. Hopefully that'll be pushing through this talk as well. Then the second part will be at least talking about the how. How we actually add semantic structure using GraphRAG, so really utilizing entities and relationships and concepts. Then, at the very end is how do we actually use this with agentic workflows as well. What does that actually mean in terms of knowledge graphs, GraphRAG, and then agentic workflows? That's the narrative that we're going to be driving today.

Part 1: Traditional RAG - Where Does it Break Down?

The RAG promise. I think everyone should probably know this workflow, but we'll repeat it just for context's sake. The basic workflow that when we started off with this really wonderful RAG and everybody was talking about this retrieval-augmented generation was this basic workflow. It's, how do you query something using natural language? What did I ask LLM today? Yes, how do I get here? What was the best route? With the snow, what am I supposed to wear? The type of queries that you're actually going to be asking from the natural language point of view. Then, what the normal thing is actually using vector searches. They would chunk up these different contexts and things like that and put it into context, which is the next thing. RAG was really about looking at similarities between the different chunks and trying to see, what actually matches and what doesn't match, those types of things.

You would feed that context into an LLM and then they would reason over that, and they would say, look at all these similar chunks that you've given me and this question, and I'm going to give you back an answer. I remember when we first started looking at this, it was super impressive. There was a lot of information out there that it was able to find similar answers and things like that. Then we very quickly started looking at some of the limitations. What happens when these queries get incredibly complex? If you think about your enterprises and all of the different questions and answers that you have to ask, it gets very complex very easily with a large amount of data. What do we do then? As I said before, when I talked about this, I had to split this up into two slides, because I would say, where did traditional RAG used to struggle?

These were the top things that I had when I wrote this talk, or at least part of it, is a few things. One is around connecting the dots, so being able to traverse relationships between different pieces of information and to really synthesize those insights. Traditional RAG was pretty bad at that actually at the time. That was actually very hard to show what is the relationship between this thing and this thing, and why do we get to that conclusion? Traditional RAG was also very locally centralized. It didn't think about the thematic themes of the actual context itself. It would try to answer these specific questions without the global context. If you asked it a specific question, it would give you that specific answer. If you wanted to take a step back and say, what does that mean in this ecosystem? It had a really hard time doing that.

The third thing was around multi-hop reasoning breaks. How do you actually look at different hierarchies that go multiple steps up? How am I related to Wes, to Aaron, and this and that, and who talks to who, and what does that LinkedIn relationship actually looks like? It broke when it went into multiple hops as you're trying to retrieve some of that information. That was actually an issue, again, back in the day. Then, of course, we talk about provenance and explainability. How did you get to that conclusion? What were the things that you looked at and the LLM was fed to, to actually come up with a specific answer? This is the traceability. This is the auditability. You think about your financial institutions and the private data they have, they'll have to actually explain, how do you get to an answer? This provenance and explainability is pretty important for retrieval.

Then, of course, the very end is enterprise domain complexity. These large enterprise with finance and healthcare and legal, if any of you are in these industries, the sheer amount of data that's in there, and how complex all of these different rules that you have to traverse and find an answer from is actually quite difficult to bring all together into one area.

I'm going to fast forward to today. I would say that from the enterprise and provenance point of view, I still think that traditional RAG is lacking quite a bit, as we can see. I think the models that the LLMs are using today are getting much better at connecting the dots and the multi-hop reasoning and the global questions. The reason being is because the context window is so much larger now. We just keep feeding all the information so they have a lot more things that they can actually take in. For example, I can feed it like 10 different documents and there's a multi-modal approach where here's a video and here's an image. Go and try to figure all of that out together. There are some improvements when it comes to connecting the dots between these different documents that you're sending it. The multi-hops, they can actually probably traverse a little bit further now, given the maturity of these models.

Then the global questions still fail, but the more context you give it, the more thematic you can actually think about these kinds of things. Again, it's only regarded to what you're actually giving it and it just still takes a sample set off of that. Hopefully this talk is still relevant because we still have a few things that we need to improve on. My company and my customers actually use GraphRAG in order to do that. We'll get to that now.

Part 2: GraphRAG - Adding Semantic Structure to Retrieval

Before we get into GraphRAG, I want to give a little context into what a knowledge graph is, because a knowledge graph is the foundations of how we at RelationalAI do GraphRAG. There's different types of GraphRAG that people can do. I'm going to explain the one that we do. Really a few things to know about a knowledge graph. Everybody always thinks graph technologies, nodes, edges, and all the connections between them. The difference between a knowledge graph is the semantics that you're actually putting on the graph structure in itself. This is the logic. These are the business constraints. These are the things that are connecting the two entities. That relationship is now full of semantics. As we all know about LLMs and AI, the more context, the more semantics you give something, the more accurate it actually can be. That's the core principle here. How we structure our knowledge graphs is between these concepts and these relationships.

The way we do it with our knowledge graphs is we use something called graph normal form. If everyone thinks about first normal, second normal, third normal. This is an example down here is that we make these very long tables as opposed to very wide tables. This is how we traverse theory knowledge graph based off these long different relationships. It's usually two concepts or two entities and a relationship that we're actually denoting down here. The reason why we do it this way is so we can capture these rich semantics, so these decision-making abilities are much better for the GenAIs and the agents themselves. Then, one of the things I will talk about as well is like now that you have something formulated in a knowledge graph, then being able to reason over these knowledge graphs become much easier. I'll talk about what that could potentially look like after this.

Then just pushing in here as well, because just to give a little context into our product, our product is actually based in the ecosystem of Snowflake. Anytime you think about a knowledge graph or anything like that, you can actually have it in any type of data warehouse or a data system like Databricks. Google's had their own knowledge graph, Amazon, all of these different types of things. For your context, I talk about Snowflake because that's the ecosystem that we've built all of our knowledge graphs on. I'm not pushing Snowflake. I'm actually pushing the fact that we like the architecture around having data governance, having scale around the knowledge graph in itself, and being able to have this security boundary. In this particular architecture, we're able to create a knowledge graph on top of the data within Snowflake, and you never really have to leave Snowflake. There are some examples of where you would take your data out of Snowflake, or Databricks, or whatnot, and you would actually build a graph next door, like for Neo4j or things like that, which is not wrong or not a bad thing. One of the things we like to do is bring the knowledge graph, the semantics closer to the data versus taking the data out, and then actually doing some processing on that. It's just a different methodology. It's not right or wrong, but this is the way we like doing is pulling the intelligence to the data in itself.

Let's get to what the title of the talk is. Now that I've explained a knowledge graph, GraphRAG for us uses a knowledge graph by extracting the entities and the relationships from the text that you give it. These are the things that you can actually build these hierarchies, these communities, once you get a really large graph, then you can actually leverage all of the different structures and the relationships and the traversals for the retrieval in its own right. Again, I've talked a little bit about where I believe traditional RAG is lacking. I think provenance is a really big one. How do you actually go from A to B or A to Z and understand how you got there, which paths you got there. This is something that is really important that a knowledge graph can capture because you can actually see the nodes and edges and the path traversal that it does.

Then, of course, deeper reasoning. As I said, the difference between a normal graph technology and a knowledge graph is the semantics that you put on the knowledge graph in itself. Because of that, we can actually have a deeper reasoning between these entities and between these things you're trying to retrieve. Some concepts I talked about. I'm going to pivot a little bit to a toy model, just so I can set it up for a demo a little bit later. I want to ground everyone in a model that I have been using in order to present this for you guys. This is called a Jaffle Shop model. I stole this model essentially from DBT. DBT uses this, if anyone's ever used DBT and did their starter kit, they have this Jaffle model shop. I've been trying to extend this into a knowledge graph. Essentially, a Jaffle Shop, pretty simple, has products, has stores, has supplies, has customers, has items, has orders.

This should be pretty straightforward. Retail, like a food shop for you. A Jaffle is a sandwich and you can have savory sandwiches and sweet sandwiches. I was like, ok, great, whatever. It made for giving a lot of good data. These are some examples of Jaffles. I have never eaten one of them, but for this exercise, it'll be good for you to contextualize that.

The starting point for us was to really create a knowledge graph based off this Jaffle Shop. As you can see to the right over here, and very not clearly to the left, I'll explain what that is. To the right is what I'm calling the ontology. The ontology is the basis for a knowledge graph. This, if you look over, you can see that before I had products, stores, supplies, customers. Now I'm giving fidelity in relationships between these stores, these customers, these order items, and what is the different business logic between all of these entities in itself. Now you're actually describing the business model in this ontology. This ontology will then represent the knowledge graph that I talked about a little bit earlier. Here is an example of the Python code that we write in order to create these relationships. Here's an example of an order. You can see, this is my Jaffle model, and I have these orders, and I have products, and I have store locations.

You can see here that I'm actually using Python to create an order and to put the different properties on there. This is actually a relationship that I've put on there. An order, is it a drink order? If people notice, is it a drink order, is it a food order? That typically for me was logic that was put in the application layer somewhere. If it's a drink order, do this. Else, it's a food order, do something else. Here we're actually putting this on the data, on the knowledge graph and the semantic layer in itself. That is pushing the semantics much closer to the data, because this is now your source of truth. This is the paradigm shift I'm talking about in terms of a knowledge graph, is I'm putting that very simple piece of logic down, a lot further down.

As you can see then, I've created this knowledge graph based off of this Jaffle Shop example of model. I have these very structured tables, and I build that up, and I give it some logic, and this is how I do that. Then on the left-hand side, I could probably ask these questions now on top of my knowledge graph. What customer X ordered 5 times? Store Y had the lowest revenue. Did customers A and B shop together? These are structured questions I can actually ask upon my knowledge graph now. Then on the right, how do I start asking questions like, is the customer X happy? Or, why is this store struggling? What is the context behind that? Because the structured data doesn't actually have that right now. Hence, this is where we want to introduce GraphRAG as part of our pipeline into building the knowledge graph.

One of the things that I've done in here, and hopefully I can demo it a little bit later, is I ingested five documents of feedback about Jaffle Shop. What customer was disappointed in which store, and what were they disappointed in? Was it the store owner? Was it the store itself? Was it Jaffle because it was too gross? What was it that they actually have feedback about? Think about this as like Yelp reviews for the Jaffle Shop. Imagine if you have years and years of Yelp reviews, how do you consume all of that data and actually understand what that actually means on your knowledge graph? Now again, going back to traditional RAG, I could probably give a page of reviews and that LLM is pretty good at deciphering what's going on there. Imagine giving many pages with the reviews and having it traverse and hop through all of those different things and make sense of it.

Our solution to this is to take those documents and start pulling out the entities, the relationships, the concepts, everything I explained to you in the knowledge graph, and amending the current knowledge graph that we have, and iterating on top of that in order to expand your knowledge graph with all of this new information that you're grabbing from this unstructured data. One of the things we've done is we've taken the customer feedback and email surveys. You can actually use anything unstructured and throw this in here into the pipeline that we have to put in the knowledge graph.

This is an example of extending that knowledge graph with GraphRAG. Again, I started very earlier with the Jaffle Shop knowledge graph. You saw how I built my model, my orders. Then I added a feedback PDF. In this case, I added six documents worth of PDF feedback, which I think ended up being like 3,000 lines each PDF. Not very big, but not that small. This is an example down here about some of the customer feedback summary. Then the GraphRAG workflow extracts those entities from all of the different documents. Then it actually creates that hierarchy. I'll explain what that actual workflow looks like. Then in the very end, you have this extended knowledge graph that has now added this unstructured data, so you can now query on top of that, on top of the original queries that you could have done from the beginning. This is the one.

This is the GraphRAG pipeline that we use. It's very important to call out here is that either you already have an original knowledge graph that you want to amend, like I'm showing here, or maybe you're starting from these unstructured data and you want to create a knowledge graph from this. This works in both directions. Essentially, the first step is to extract the entities and build a hierarchy from these entities from the documents, so it will pull out all of the different things. In this particular customer feedback, it pulled out the different names because those were the people giving the feedback. It pulled out the feedback as an entity. It created a relationship saying this customer has given this feedback at this particular store about this. This is the graph or the traversal that it's actually creating. I want to note that there's a circle here because it is an iterative process.

It'll keep going back to the documents. If you keep adding documents, it'll keep amending the knowledge graph and relooking at the different entities and nodes and try to reconcile that as it keeps going over and over again. This is the process flow that you're following. It re-extracts entities, iterates. It uses the ontology that I showed you, and amends it, or it actually reconciles. It's like, is this an entity or is this an entity? I see this many more times, maybe it's a new entity. It actually starts learning on top of that. Then it actually builds on top of that ontology that I showed you and extracts all of the logical expressions, the validations, and everything from those documents, and enters that into the knowledge graph in itself. Then by the end of that, you should have a knowledge graph that you can now query on top of and to get more information, not only what is the top store with revenue, but who was upset about that store in the last three months and why, and who recommended that.

Again, I'm in my Snowflake ecosystem right here. Essentially, I did stage the files. There are six files that you would see that I've staged in here. To the left here, you can see that there are six different files that I've uploaded. You can upload hundreds of files in there and take it through this pipeline. I did it for this purpose because, how much feedback can you really give a Jaffle? There's a lot of feedback here. Then, of course, these are some of the tables that have that extraction process. We actually did the corpus, so took the big document and then started parsing out all the edges and nodes and entities and things like that. This is an example of what we did in Snowflake to actually pull out some of these documents.

Then, of course, this is now the next step. You can actually now do much richer queries on top of this extended knowledge graph. Again, as I said before, a GraphRAG, we could actually ask some questions. Which store had problems? Who should we target for referrals? They don't really know how to actually answer them. If you actually ask here, which store has complaints and why? It now actually has more information that it's extracted from these documents. This is a pipeline that you can actually put into your workflow and automate this entire process. Every time a new document comes in, you can actually send it through the pipeline and it will amend and extend that knowledge graph. A few things to think about here. If you think about, I'm trying to find information on this feedback, you could send these documents into ChatGPT, or Gemini, or Claude, or the LLM of your choice.

Then, how does it keep that history over millions and millions of documents? This is where you can actually have persistence in that knowledge graph and iterate on top of that in order to grow a knowledge graph, not just to have in one state and time, here's all the documents, give me something. It will give you something interesting. How do you do that from an enterprise sense? You want to keep that knowledge graph building and growing to give you a lot more information, history, provenance, all of those things.

Part 3: Agents, Decision Intelligence - From Passive Retrieval to Active Problem-Solving

Third part is around, how do we use all this? If I think about my customers right now, they will look at me, at the GraphRAG and say, "Great, you build a knowledge graph, Cassie, but what can I do with that? I'm an analyst. I don't want to build this thing. You've automated this thing for me. I just give you documents. How do you actually use agents and what does that actually mean for this?" Again, beyond retrieval, if we think about the agentic workflow, agents, these are the capabilities. Agents have memory. They have short-term and long-term context. I actually just discovered like the, I don't know if it's a workbook or worksheet, or I have a project and then I can just keep feeding it documents and I can have a history and they understand who I am, and I'm the audience and what I'm looking for, that type of stuff.

Now it has memory. It also has the ability to call APIs, databases, functions. It can actually utilize all of these different tools that you give it. That's a great thing for an agent. It can actually plan a lot of multi-step workflows. I think about, how do you send an email, or how do you do all of these types of things? There's a lot of talks I know coming forward around, what do these workflows actually look like? If there's a human in the middle, if you're automating the entire process. What that actually looks like. Then, reflection. How do you actually take something and say, that wasn't right or that wasn't quite correct. Let me reflect on it and actually look for different answers and understand what that looks like. These are agent capabilities. Sounds really great? No, not necessarily. I think it's the way you use the agents that we have to really think about.

For memory, the memory is there, long-term and short-term memory, but memory cannot guarantee the correctness. It literally just preserves the state that you gave it. Here, I have a state, this is my memory for it, but I can't tell you if it's correct or not. I just can tell you what you've given me, so that's the memory of it. Let's not conflate memory with correctness here. The tool, an agent knows when to call a tool, but not how to reason about that tool. It may not actually know when or why I want to use that tool, or if there's two very good tools to use, which one should I use? An agent doesn't do that. These are the directions that we have to give the agent in itself. Remember, agent's not this decision-making powerhouse. You are literally feeding it tools and things in order to do something with.

The planning, this is all about control flow, not domain intelligence. It's not going to change that control flow and that workflow that you gave it, because it's like, I'm getting smarter, so I'm going to actually change my mind about this workflow. It's going to use the workflow that you have provided and you are working with these agents to do, and agent to agent or MCP, you're actually telling them what to call and when to call. Then the reflection piece is really important, because if you keep feeding bad information, it will keep reflecting on bad information. Reflection needs ground truth. One of the most important things I think about when I think about these agentic workflows is ground truthing it all the time and actually driving evaluation frameworks to understand if your answers are right, to understand how accurate the agents are actually driving up these workflows and iterating on top of that. Ground truth is incredibly important to think about when you're looking at reflection.

This is an example of what we use for our decision intelligence and how we use agents. Again, I'm in the Snowflake ecosystem because that's where we are, but you can use any type of ecosystem when you're thinking about these types of agents. More specifically for us, we use what Snowflake has is called a Cortex Agent. For us, I will show in this demo around how it uses some of our RelationalAI tools and the things that I give it, as I talked about tools, in order to actually reason upon something. If you can look at this actual architecture here, an agent, you can't just say, agent, go and do whatever. You have to give it these tools, these reasoners that we give it here. For our technology, we think about reasoners as like, how do you ask a SQL question? How do you do similarity research?

Even more importantly, with a knowledge graph, we have the ability to actually reason over rules. These semantics allow us to actually look at the different rules within the knowledge graph. I think for folks who've done Neo4j, you can actually have a lot of cool graph algorithms like PageRank, WCC, all of these types of things. Those are reasoners that you can use. The other reasoners you can use are prescriptive and predictive. We're thinking about prescriptive reasoners. I'm thinking about supply chain. If this warehouse goes empty and loses all of its supply, or there's a weather storm or whatnot, how do I reroute all of my travel to another warehouse? That's called prescriptive reasoning. There's a lot of different solvers and mechanisms that we do here in terms of the reasoners. Then, of course, the predictive reasoners, like graph neural networks. How do you know what's going to happen in the future based off what's happened in the past?

These are the types of reasoners that you can actually use now that you have a knowledge graph to traverse all of these different types of things. If I look at the different layers, the semantic layer and the reasoners are literally the core foundation that we are putting together. The agents are just on top trying to orchestrate all of these types of things. I would inverse the thought process if you had, of agents is going to solve the world, more the agents are your orchestration piece, but everything down here needs to be solid. If you have dirty data, if you have data that's unfungible and you can't make sense of it, you can put every agent on the world on it and it won't be able to do anything with it. I always think about foundational pieces. How do you structure your data? We structure ours in knowledge graphs. Even if you structure it in a clean way, like I talk about DBT, DBT does great transformation to clean up your data. If you think about those foundational pieces, then being able to utilize agents, whether it's Snowflake Intelligence, or Gemini, or Claude, or whatnot, that's what's going to be the powerhouse is this foundational piece down here.

Demo

Demo time. What I'm going to show you is actually a workflow from using Snowflake Intelligence and using these agents to ask these questions. As I'm talking through this talk, we have a knowledge graph. In the last few weeks, I've built and pulled in those documents that I showed you and actually took it through the GraphRAG pipeline. Then I have some questions that I could ask based off of those unstructured data. Then I have a question that I could ask that is around finding friend groups off of ordering patterns, which is utilizing a graph algorithm called Weakly Connected Components. These are very common algorithms that you can use, but these are things all in the ecosystem of Snowflake using those agents.

Let's try this. "Good morning." It's still morning. Wonderful. What you're seeing here is Snowflake Intelligence. Everything that I've just showed you is behind the scenes of this agent in itself. As I showed you here, let's go ahead and ask a question. I'm going to ask just a basic knowledge graph question, tell me what semantic models are available to me. This is based on, what did I feed this? What is in the ecosystem that I'm actually looking at? If you think of you as an enterprise, you can actually feed its own model. This is in its own perimeter. You can actually stop it from going out to the internet and actually asking different questions. You can actually say only ask the questions of the domain and the knowledge graph that I have today. That worked. You can see, as we're talking about that, we have this Jaffle Shop knowledge graph.

What's really great about this is because I fed it a knowledge graph, it now can tell me all the key concepts, the customers, the products, and some of the available queries I can actually do on this knowledge graph in itself. I will talk about the ontology, and let's see if it comes back with something. If we look at the ontology of the Jaffle Shop, how do store locations relate to other order items? Very simple. Could you imagine if you think about your domain right now and your codebase, if you ask a question of how does this thing relate to this and what's the business logic behind that, wouldn't that be great for your product owners and your business to actually be able to codify something like that? This is why I really love having all these business logic and things within the knowledge graph, because it's right here.

Now I can actually query. I don't have to pull it in from all these different documents, in Confluence, in GitHub. I can actually pull it in from this knowledge graph and understand the providence. It shows here the detailed relationships. It actually shows the store location has an order, which has an order item, very simple. Again, think in the context of where your domains are and how complex it can get. You can have order1 that's placed at a store location. A lot of really great information about how these two entities and concepts are related. It actually looks at the indirect relationship, and I could probably even ask, how did it get to that indirect relationship, and show me the traversal of how that happened.

GraphRAG. Let's talk about GraphRAG. From before, we probably couldn't answer this question, but this question is around, for each of the five problematic stores, list the staff members who were mentioned in negative feedback. Which store had the most complained about staff? That's a pretty complicated question. Being able to extract the feedback that I had from those documents, it actually came back with quite a few things: Philadelphia, Atlanta, Portland — Atlanta had a lot of complaints — Detroit, LA, even worse. These are some of the answers that it's getting from the knowledge graph that we extracted from the documents themselves, and it actually can say how these things are related to each other. Maybe the question is, and I'm pre-looking at these questions, but I'm trying to show the different multi-hop steps that you can get from who's referring Jaffle Shops from where and when, and all of these types of things.

This is yet another example of a question I can ask based off of the GraphRAG pipeline that I ran. It actually can follow the chain of multi-hops of who referred who, and did anyone in that chain submit negative feedback. Here's a 10-hop chain that I asked for. This person referred this person, and so on. Then it can actually tell me which one of those people actually gave negative feedback. A lot more information and fidelity that I'm getting from these unstructured data that I've now put into my pipeline. Then I could actually ask this, this WCC algorithm. I think this might take a little bit long. This is a query that's going towards our native app, our RelationalAI that has a library of these reasoners. It'll take WCC or the more known graph algos, PageRank that Google uses, and it takes that algorithm and it traverses your knowledge graph and it finds information about communities. That's what this particular algorithm is doing. That is the demo.

Key Insights

I think traditional RAG still has its limit. It doesn't have as many limits as I was stating six months ago, but it still has some limits. One of the things I think about is the evolvability of change. The change rate is pretty high, but it's really back to that last slide about foundations. How do you actually structure your data and have robust and reliable pipelines that will get your data in the right place so you can actually adapt to all the new models that are coming, to the new technologies that are coming, the new AIs that are coming. That's actually the most important piece that I want you guys to take away from GraphRAG as a tool. I don't think GraphRAG is a solution. As you can see that I did, GraphRAG is one of the many tools that I have in here that I'm using my agent to call upon, depending on what context you have.

Do you have a lot of unstructured data? Maybe you can use that pipeline. Then, of course, agentic workflows unlock multi-step problem solving. Remember, the agents aren't your problem solvers. You are. That's what Hilary said, is like, you guys are the problem solvers here. You're just using agents to make you way more efficient at it. You still have to feed it logic. You still have to feed it the tools. You have to give it its memory and you have to give it its ground truth.

Questions and Answers

Reisz: What was translating that question back? What was the retriever that was pulling information from the GRAG? Was that a MCP talking to an API? What was actually translating that text?

Cassie Shum: What was translating that text? It was not MCP. It was actually an LLM that was actually taking the text off of the documents themselves and throwing them into the nodes.

Reisz: On the chat interface.

Cassie Shum: The chat interface? That is actually MCP. It's Snowflake Intelligence, or you can actually hook up that MCP protocol to Claude or any other agent and ask the same exact questions. Then MCP, of course, is the protocol that opens up these tools. It makes these tools discoverable, so you can actually ask all the different questions and things like that.

Participant 1: I'm all over GraphRAG. I'm just getting used to all the terminology. I hadn't come across the reasoners, your ground truth. I understand what it means. Can you tell me how that works? Where do you put that ground truth so that the facts are there to be reasoned and corrected?

Cassie Shum: You're more asking where those reasoners actually lie and how you're able to use that on the knowledge graph in itself?

These reasoners here are essentially either libraries or different protocols. These reasoners come in different shapes and forms that are bundled in our product. Our product in RelationalAI, we're bundled in a thing called a RAI Native App. If you think about, Snowflake has a marketplace like iPhone has these apps. We have an app on the Snowflake ecosystem, which actually bundles up all of these different reasoners. Right now, we have solver-based reasoners, which is that prescriptive supply chain I was talking about, and then a whole library of graph algorithms and then predictive reasoners. What happens is, is that once you actually build your knowledge graph within Snowflake, you're literally using our app in order to invoke the reasoners on top of that knowledge graph and then writing it back to a Snowflake table so you can query that Snowflake table. That's that flow piece there.

Participant 2: Do you find that it tends to be better to try and get all of your data into Snowflake? For example, you had your PDFs with the ratings, or you had some business logic in the Python. Is there a happy medium that you found with some of that business logic living in the code versus the data, or do you try to get it all into your data sources beforehand?

Cassie Shum: It definitely depends. It depends on where your data started. I don't think you should be using all of your time trying to put all your data into one place just to make this work. This is why componentization in these different workflows are really important to understand, because you can actually use these PDFs and extract them in another pipeline somewhere and then bring that graph into Snowflake or to anywhere you want. It's really about the workflow and not the actual all your data system in place. I care about that only because a lot of our enterprise customers that we have already have their data in one place, which is Snowflake. Their biggest requirement to us is, don't move my data. I want to do all these really cool things that you're talking about without moving my data. That's only one subset of my customer. With those kinds of customers, yes, this is actually quite important. With others, like when you have these pipelines, these data workflows and things like that, it's really about looking at that cycle time and like what I was doing there, building those workflows out however you can.

Participant 3: I understand the business perspective. This is awesome. At my job, we start to document all the architectural decisions in Markdown and start to save that in Git repositories. How is the application of this for, for example, improving the decision-making in technical decisions, make change, and how this affects the architecture of the company?

Cassie Shum: Simplistically, you're asking, how does this increase decision-making, essentially? For this particular architecture, you can utilize multiple different reasoners and you can look at your business logic on top of your knowledge graph. I think that is the core. The semantic layer is a core of why decision intelligence is much higher now, because instead of actually having to go into your application layer, to these PDFs here and to this, you're trying to bring everything together. In this particular example, in the semantic layer, I've pulled everything into the semantic layer and the business logic so you can reason on these things in one holistic state. Referring back to the question earlier, those data sources can be from anywhere. They don't all have to be in Snowflake, but what's the binding piece here is that semantic layer knowledge graph, that ontology that actually builds those relationships between the different nodes and entities is what gives you more fidelity, so you can actually use these reasoners.

Participant 4: I have a question about the storage. From what I read briefly about GraphRAG, it seems like a Python library, but where does it store the data, actually? Is it just in memory only?

Cassie Shum: In Snowflake, they separate storage and compute here. You could store as many documents in here as possible. Snowflake will only charge you for the compute, so only the processing of these documents and the query time of these knowledge graphs is what they're actually charging you for. Storage, I don't want to say infinite, but they proclaim infinite. You can actually put a lot of storage on here without being charged that much. That's why a lot of big enterprises go here, because it's really about computation on the data and not the storage in itself.

Participant 4: Is it possible to connect to classical graph databases, the SPARQL query language?

Cassie Shum: Yes, like typical graph databases like Neo, or SPARQL, or anything like that already have the structure of nodes and edges and things like that. What it lacks right now is the semantic bits on it. As I said before, Google uses a graph database that actually uses PageRank, for example. You can still utilize some reasoners. I think I like knowledge graphs more and I'm supposed to because it's my product, but it's because you can actually utilize more than graph reasoners on top of it.

Participant 5: I have a question about extending the ontology in the Jaffle Shop example. I understand how as we take in these reviews, we find structure from the unstructured data and you get, say, instances of a customer. Person A was a customer and we had defined what a customer is. Do you at any point in that pipeline come up with the idea of what an influencer is? Like at the beginning, you don't have that defined. Then you process and you get a sense that people are talking to others. Are you generating and basically defining that new entity at one point?

Cassie Shum: One of the things that we do on the GraphRAG pipeline, I didn't put it in here because I didn't want to complicate the situation, is that we actually use our reasoners in order to identify different nodes and entities. Your example of an influencer, I've actually run some of these graph algos on top of the knowledge graph that we're seeing. We're like, this customer is connected to these 10 customers. That most definitely is an influencer. Now we've created a new concept of an influencer based off of that reasoner. We've used a lot of our technology in order to create more fidelity in that knowledge graph.

See more presentations with transcripts