正在加载视频...
视频加载失败
Building a RAG system that works with real-life documents is crazy hard. Why is nobody talking about this? (I'll show you what a complex document looks like in the attached video. Good luck with the 10-line code demos if you want to deal with this.) All I see online... show more
10 条评论
have you seen looks similar and promising?
I haven't seen it, no. Thanks for sharing it!
We agree. That's why we build @trustgraphAI. TrustGraph is a full model agnostic AI Engine with native GraphRAG. No need to build anything. Deploys with Docker or Kubernetes.
I thought that building RAG systems is easy until I tried to build one. If you want it to work well it is really hard.
Yeah, building a demo is very simple. You can understand the basic components in a few minutes. But doing anything serious with it is really complex.
Really amazing! How can one find the documention to install it in your own Kubernetes cluster and run it locally? I've only been able to find the SaaS option...
The problem is, in the most of the enterprise use cases, where the data is highly regulated, they do not approve open source frameworks.
Actually, I've seen two different approaches: 1. Enterprises that don't approve closed-systems running outside their premises. They would rather use open-source. 2. Enterprises that don't approve open-source software that doesn't come with certain guarantees or is too complex to set up.
Or just pass your pdf's to gemini files API. Is working very well for me with 15 documents, with 10 to 165 pages each, all in a single prompt. Gemini finds and extracts multiple small details and builds and returns a compliant JSON response.
If this is working for you, that's awesome. But in many cases, relying on the output of the model is not enough to build a reliable retriever. Notice that here, the process is not just about "understanding the document," but augmenting it with contextual information as well.
![Vallée and the Closed System: Are We Prisoners? 🧠👽 Vallée: "Are We Being Taken Over by a Species from Somewhere in Space That's Vastly More Intelligent Than We Are?" 👽🧠 "..the simulation...was a new concept that I initially rejected." ~Vallée "Is it looking for us to try to interact with it as equals or with parity?" ~Scafish "Nobody says that to Congress, and I think Congress should hear it." ~Vallée If, "it's a closed system, we're like prisoners and something is going to happen to us, and there is very little we can do." ~Vallée Turn the thermostat dial. "If the temperature doesn't change, then I know I'm inside a control system. So, we can do the same thing with UFOs, but we have to react. We have to, number one, acknowledge that it exists, and number two, we have to react to it." ~Vallée ~~~ I've been wanting to share this one for a while... Vallée: "So, he said, the question you have to ask about UFOs is, number one, is it a natural system or an artificial system...control system. And if it is a control system, is it open or closed? In other words, are we being taken over by a species from somewhere in space that's vastly more intelligent than we are?" (I've never heard him even suggest that possibility.) Vallée: "You know, as Dr. Garry P. Nolan says, you know, people who have had ten, you know, scientific revolutions, or a hundred or a thousand, and come here with superior science to do something... And in which case, you know, it's a closed system, we're like prisoners and something is going to happen to us, and there is very little we can do. Or, is it an open system where we can, in fact, communicate with it. And if we can communicate with it, then the question for me as an information scientist is, what are the modalities of the interaction, you know? It's not just can we learn their language? And they say, you know, 'We come in peace to save mankind,' or something. Or 'We will give you the cure for cancer' or something. I don't think it's at that level." (Will we ever be able to get answers to these extremely important questions? If it's an open system, how do we communicate with it? How do we provoke it to react? We know it reacts to anything nuclear but we still don't know why. This is why we need the USG (and other governments) to present evidence that shows the masses this is real and extremely important for our species to investigate. If that evidence exists and is shown, we'll have an easier time getting the world's best minds to join the effort in figuring out the best way to answer these questions. We still may fail but we should at least try.) Vallée: "I think it's a meta-system. It's not a system. And that's my fear...if we can circle back to your earlier question about, you know, about NIDS and about BAASS, what we did for the government and what we did for the Defense Intelligence Agency. Half of the budget was spent developing, you know, a super database. And we don't know where it went. I mean, I'm not cleared to know where it went." (On the contractor (BAASS) side, Bigelow should have all copies of what AAWSAP produced. And on the DIA side, Lacatksi said he put all of the digital files in a specific place that he didn't name. As long as someone didn't delete it all, it should still be there. Vallée has said that the Capella database has about 250,000 cases from around the world.) Vallée: "But that would be a very interesting question, because the people who are getting [the database] are getting raw data, which we have very well organized, all in English. So they have the luxury of, you know, we had five translators from French, English, Portuguese, Russian, you know, everything was translated in a single structure across fourteen databases. "That's what we need to answer the question about the control system, and it's not being done. And we hired a whole team that we had trained to work on it. So to rebuild that will take the next ten or fifteen years. And nobody says that to Congress, and I think Congress should hear it, because it's our money." (As long as names and personal details are scrubbed from that database, there is no reason NOT to release it to the public. This way, we can take it and use AI to help decipher patterns and maybe answer some of these questions. Can Congress help us get access to that database?) Vallée: "When you ask, is it a control system? That's a big question." Peter Scafish Peter Skafish - "You asked, at one point, whether the system is open or closed, and you said, additionally, I believe, if it's open, that it would be possible to communicate back to it. And it sounds to me like that's the key question for you. Is, if you can understand what what the system of symbols is, or the modalities of communication, then you can understand enough to engage in some kind of communication, or at least give some kind of response to show that you understand." Vallée: "Yes." Scafish: "So then the question, and we have a member named Jacqueline, who has asked this. Could the system be stimulating us - provided there is such a system - to interact with it, more as subjects or agents than as something like animals or objects? Is it looking for us to try to interact with it as equals or with parity?" (When people report getting injured or sick from being in close proximity to UAP, it suggests to me that the phenomenon won't go out of its way to avoid affecting us in a negative manner if we get in their its/way (assuming it even knows that close encounters with UAP are not good for humans). Kind of like how we treat lower lifeforms. If we encounter a wild rabbit crossing the road, many of us will do our best to avoid it, but not if it means damaging our car or ourselves in an accident. It may ruin our day if we hit it, but it won't stop us from driving again in the future. Do NHI have bad days if their tech injures us or makes us sick? I have no clue.) Vallée: "Well, what I saw in the notes you gave me, is she was also asking: Is it a control system because we think it is? And that's a very interesting question. Because we react to the UFO phenomenon, or the UAP phenomenon. And, you know, at this point when I think about what I'm going to do next in this research, if I'm given the the opportunity to live a little longer, I'm not going to go back and write any more computer programs. There are better people to do that now, they have the data, and we're in a different phase now. We're in a whole different system. I have the luxury of doing some experiments I wanted to do for a long time." (Would have liked Scafish to ask him: What types of experiments?) Vallée: "So, if you think you are inside the control system, there are things that you can do. Or, if you think you're inside the simulation, you know, which was a new concept that I initially rejected, and then, you know, Ray's (Kurzweil?) work and others have brought it back to the forefront. And we have to ask that at the same time. Can we test it? How would you test it? Well, if, you know, I'm here in my apartment, and the temperature is constant in this apartment. But outside, I can see it's cold, or I can see the sun is out and it's warm, and how come it's constant here? So this would lead me to think that there is a control system, namely a thermostat, that is somewhere. "So I can start looking around the walls, and if I see dial, I can turn it, or I could start a fire and see what happens, see if the temperature changes. If the temperature doesn't change, then I know I'm inside a control system. So, we can do the same thing with UFOs, but we have to react. We have to, number one, acknowledge that it exists, and number two, we have to react to it."](https://image.24vids.com/tw-1999241328388112749/amplify_video_thumb/1999240752300498944/img/-ITml6B_VyoovXQH.jpg)
![I hear so often from the Dommes I work with that they struggle with people online fetichizing them and simply seeing them for how sexy and beautiful they are. They project their fantasies and their desires onto you. That stops immediately once you move the attention from you to them. From 'look at me' to 'I see you'. What does that look like? When you create content, think of them and what this scene or that narrative is evoking. What will they learn from you? What they want is not to passively watch how sexy you are, but for you to train them, to give them instructions, to teach them, to guide them, to be in charge, to command them. This is not being an object but the main subject. The Authority figure. How is your content already doing that. The sexy photos can still be there, they are important to already capture des attention. But what you do with that attention once you have it, is where the power dynamic is established. Positioning yourself as more than a stunning Goddess, but actually a woman who has a voice, opinions, perspective, a philosophy, a way to doing things, teaching them what you like, how you like it, why you like it, already makes them want to be that for you. You hold the attention, you hold the power, so you direct it. And for that, you want them to know you get them and you know what lives within them... that creates the desire for you to be the one exposing it. You instantly build trust. Not because you demanded it, but because you earned it: you showed them you know what you are doing. You have experience, you understand them. They are not told to come see you, they are seduced into it. They desire it. And they will work for it. This will attract better clients (real subs) and instead of you trying to get their attention, they will work to earn yours. If you want to learn more about power dynamics, building a brand as a Pro or the psychology behind BDSM, you can now access all my trainings and classes in one place for a fraction of the cost of The Dominatrix Academy. And you can reinvest the total amount towards the Program. Message me [SECRET] for the details. This offer is not available on my website.](https://image.24vids.com/tw-2049167429041824109/ext_tw_video_thumb/2049167371538251780/pu/img/25DXjHkj6hG_xBzD.jpg)
