One cool thing about ColBERT-based search compared to the... cosine-based vector retrieval is that you get interpretability for free as a byproduct of the MaxSim computation. It's kind of like the Lucene highlighter, letting you grab the most relevant snippets from a long document to show users where their query matches. With Jina-ColBERT-v1, which supports up to 8K token length, released by us earlier this Feb., the visualization of the late interaction between a query and a document is almost... artistic. The video shows the late interaction between the query "Elephants eat 150 kg of food per day." and the Wikipedia article about "Indian Elephant". Darker colors indicate stronger semantic matches. The darkest area corresponds to "The species is classified as a megaherbivore and consume up to 150 kg (330 lb) of plant matter per day." from the original article.show more

Vector Database by Hand ✍️ Vector databases are revolutionizing... how we search and analyze complex data. They have become the backbone of Retrieval Augmented Generation (#RAG). How do vector databases work? [1] Given ↳ A dataset of three sentences, each has 3 words (or tokens) ↳ In practice, a dataset may contain millions or billions of sentences. The max number of tokens may be tens of thousands (e.g., 32,768 mistral-7b). Process "how are you" [2] 🟨 Word Embeddings ↳ For each word, look up corresponding word embedding vector from a table of 22 vectors, where 22 is the vocabulary size. ↳ In practice, the vocabulary size can be tens of thousands. The word embedding dimensions are in the thousands (e.g., 1024, 4096) [3] 🟩 Encoding ↳ Feed the sequence of word embeddings to an encoder to obtain a sequence of feature vectors, one per word. ↳ Here, the encoder is a simple one layer perceptron (linear layer + ReLU) ↳ In practice, the encoder is a transformer or one of its many variants. [4] 🟩 Mean Pooling ↳ Merge the sequence of feature vectors into a single vector using "mean pooling" which is to average across the columns. ↳ The result is a single vector. We often call it "text embeddings" or "sentence embeddings." ↳ Other pooling techniques are possible, such as CLS. But mean pooling is the most common. [5] 🟦 Indexing ↳ Reduce the dimensions of the text embedding vector by a projection matrix. The reduction rate is 50% (4->2). ↳ In practice, the values in this projection matrix is much more random. ↳ The purpose is similar to that of hashing, which is to obtain a short representation to allow faster comparison and retrieval. ↳ The resulting dimension-reduced index vector is saved in the vector storage. [6] Process "who are you" ↳ Repeat [2]-[5] [7] Process "who am I" ↳ Repeat [2]-[5] Now we have indexed our dataset in the vector database. [8] 🟥 Query: "am I you" ↳ Repeat [2]-[5] ↳ The result is a 2-d query vector. [9] 🟥 Dot Products ↳ Take dot product between the query vector and database vectors. They are all 2-d. ↳ The purpose is to use dot product to estimate similarity. ↳ By transposing the query vector, this step becomes a matrix multiplication. [10] 🟥 Nearest Neighbor ↳ Find the largest dot product by linear scan. ↳ The sentence with the highest dot product is "who am I" ↳ In practice, because scanning billions of vectors is slow, we use an Approximate Nearest Neighbor (ANN) algorithm like the Hierarchical Navigable Small Worlds (HNSW).show more

Reminder: The Democrats are not an opposition party. Their... function is to sell the illusion of democracy and cover up the truth, which is that the US is a dictatorship of capital where every aspect of society from the government to the economy is controlled by and for the rich.show more

HOLY SHIT...you can "inference" LLMs in Sora 🤯 the... prompt was: "Open chatgpt and send a message!" How insane is it that the generated audio is not only a relevant response to the query that Sora made up out of nowhere, but the haiku is syllable-accurate?! 🥲show more

Unlocking Zero Point Energy. The Casimir effect is a... spontaneous force which has potential applications as free source of energy. The effect can be manifesting as a repulsive tendency between two conductive plates in a vacuum environment or can be a phenomenon of attraction between the same plates. A number of characteristics determine the attraction or repellence. The current understanding of the Casimir effect is based on the interaction between the fluctuations in energy density of the vacuum and van der Waals forces at play. The vacuum is thought to be the primary driver in the dynamical effect while the van der Waals contributes to the attraction. Because the Casimir torque arise from the #quantum fluctuations of energy or, more precisely from the zero point energy of materials and their dependence on the boundary conditions of the electromagnetic fields, these kinetic forces can be tailored, this raises the intriguing possibility of exploiting it for free energy. 🔗show more

Added context to my tiny diffusion model to enable... sequential generation of longer outputs! Currently the context is a quarter of the sequence length (seq_len=256, context_len=64). I have a theory that the less semantic-value-per-token, the worse the “curse of parallel decoding” is. With parallel decoding, we independently predict multiple tokens in one step. With the sentence “My poker hand was a ___ ___”, two valid predictions are “two pair” and “straight flush”. Because each token prediction is independent though, we can end up with a nonsensical output like “two flush”. This seems to be exacerbated with low semantic-value-per-token, as now you need more tokens to express the same concept. Instead of needing to independently predict two tokens, we might need to predict 10 instead (which is of course much harder). The model currently has noticeably worse output compared to nanogpt (similar size) and I believe this is a main reason. I’ll try adding confidence-aware parallel decoding (from NVIDIA’s Fast-dLLM paper) and other tricks and see how much they improve generation quality.show more

In 2000, Jennifer Lopez attended the Grammy Awards wearing... the now-infamous green Versace "Jungle Dress". The striking, plunging gown immediately became a global sensation, generating an unprecedented demand from users searching for photos of the dress. At the time, Google's search results were primarily text-based, and the company realized they couldn't directly satisfy the massive, visually-driven query. This overwhelming and specific search demand, which became the most popular query Google had ever seen, served as the direct catalyst and motivation for the company to develop and launch Google Images in 2001. . . . #jenniferlopez #grammys #google #tech #usashow more

Here is a 3D visualization of the coming solar... storm using model parameters from NASA's run, NOAA's run (shown below), and the coronagraph views from STEREO-A/COR2 and SOHO/LASCO. The storm is expected to hit Earth early December 9. The 3D visualization clearly shows the low inclination orientation of the solar slinky core inside the larger storm envelope. Of course, the earlier eruption that erupted to the northeast, might deflect this structure a little bit, but likely that will only make the structure move more towards the ecliptic plane, which will give us a larger chance of crossing through the solar slinky core. One nice thing about low inclination flux rope cores is that they almost ALWAYS have southward field in them at some point. So this is good news for aurora photographers-- as long as this solar storm stays true to its launch trajectory and inclination, we could see some organized southward pointed magnetic field. Considering this solar storm is fast-- that usually indicates the storm will not deviate much from its original launch trajectory! Here is the link to a static frame of the 3D frame visualization so you can fly around it yourself:show more

DOM3 is made from HTML, CSS and Javascript, closer... to a document than a canvas. It's a strange contemporary material that shapes much of our digital lives. The rhythm of endless creation and deletion in DOM3 is a self-driving version of what happens when you browse the web. "The Document Object Model (DOM) connects web pages to scripts or programming languages by representing the structure of a document—such as the HTML representing a web page—in memory." A html element, and a rectangle on a canvas, are not the same thing. It's a different material. Even if it all ends up controlling the same pixels, the process is different. The html element kind of actually exists, in this abstract context of a document object model. And in memory, in the moment. If it was possible to measure its weight, it would be heavier than a pixel on a canvas.show more

Due to a bizarre interaction between Pokeys and Turn... Blocks in Super Mario Maker 2, it is possible to create a "Jumpscare Pokey" that rises up extremely quickly and unexpectedly from the bottom of the screen.show more

Playphrase. me is a website where you can look... up a phrase and it'll show you the same phrase found in movies or tv shows. It shows you over 1000s of matches and it's completely free. This is great if you want to find funny clips for videos saying the same phrase.show more

how based are you? it’s nice to see how... more and more new apps are appearing inside the base app every day. it’s a great opportunity for users to get in early and earn rewards it’s also a big boost for the base ecosystem as a whole: active users get rewarded, the ecosystem grows, developers earn, users earn. a real win-win that’s what “base is for everyone” is all about “base me” is a fun app from the creator Fokki. you can spin the wheel and find out just how based you are. i got lucky and ended up being as based as jesse.base.eth you can join and try it yourself using the link belowshow more

Realizing people might not have realized that by "flower... game" I mean this end screen is a DIRECT reference to the book of unveiling in Destiny 2, which itself refers to a modified version of John Horton Conway's Game of Life; the book refers to this as The Flower Game, a zero-player game where the only move is the initial set up of the pieces and their behaviors. I think if this Durandal is the one from the end of Marathon Infinity, then perhaps he's found a way to influence the Flower Game, and is using that influence to re-write the universe to create a reality that allows his escape from reality itself.show more

🚨MARKETS: WATCHERGURU SAYS $150 $XRP IS PRACTICALLY IMPOSSIBLE In... a new article titled 'XRP Has No Future: What the Numbers Really Tell You', WatcherGuru points out that to reach a token value of $150 per $XRP, based on current supply... ... $XRP's market cap would have to reach $13.5 trillion. "That is around 10 times Bitcoin’s current value... This is a big part of why XRP will never go up to the levels that viral social media posts keep promising" The report does however, highlight the various major institutions that leverage Ripple's technology, as well as more feasible price predictions for 2026. This includes 21Shares' estimate for $2.45.show more

Apparently, I saw this video online and I decided... to share. What this worker is applying is called bitumen, or what many of us know as bituminous coating. Most people think a wall is a solid, impenetrable block, but in reality, it is more like a sponge. Concrete and blocks have microscopic pores that pull water from the earth through a process we call capillary action. This thick black substance is the shield that stops that water from climbing up into the house. It is not about making the wall look good because this part will be buried under the dirt forever. It is about creating a skin that water cannot breathe through. When do you need to do this? The need for this arises because the soil is a very aggressive environment. Water is not your only enemy.. The ground also contains salts and sulfates that want to eat away at the cement. If this moisture finds its way to the steel bars inside the columns, those bars will start to rust. And when steel rusts, it expands, and that expansion is what cracks the concrete from the inside out. This coating is the only thing standing between your foundation and that kind of slow destruction. Thats is why if you see wet patches at the bottom of your walls inside your house, it usually means someone skipped this step or did it poorly during construction. You can apply this anytime you are building parts of a structure that will stay in contact with the ground. It is common in areas where the water table is high or where the soil stays damp for most of the year. This is a one-shot opportunity. Once you backfill the soil, you can never go back to fix it without a lot of expense and a lot of digging. It is about having the foresight to protect the heart of the building while it is still exposed. Please don’t ignore this if you need to. If you ignore it now to save a bit of money, you will be funding the future decay of your own home. I hope this helps.show more

This isn't a criticism of Kevin Durant, but of... the Suns as a whole. If Jaylen Wells is going to faceguard Durant like this off-ball, you have to get your main guy freed up with a screen. Phoenix just sort of let it happen, and the result was a not-great shot as the clock wound down. This was just one example of a few times that this happened late in the game.show more

When you grow up surrounded by love, the boundaries... between species don’t seem to matter anymore. 💖 From the day he hatched, this little tortoise has been raised alongside a pack of dogs. He has never known a life without wagging tails, floppy ears, or the sound of paws racing across the floor. From his earliest days, he has shared sunbeams, snacks, and afternoon naps with the whole pack — and no one has ever questioned whether he belongs. To him, he is simply one of the dogs. And to them, he is a true part of the family. 🐢🐾 📹Pawbuddiz Rescueshow more

Looking for the perfect holiday gift for your baby?... How about a “face book?” I’ve written often about how much infants love the faces of their loved ones. Indeed, we’re biologically driven from birth to distinguish - and study - the faces that matter most. That’s because, as infants, the faces of our primary caregivers teach us lessons about everything from our safety to how language works. The brain has specific structures that help us to detect and distinguish between specific faces, including the fusiform face area, a part of the visual system specialized for this very task. I’ll have more to say about holiday gift giving soon (spoiler alert: for little ones, developmentally, less is more) but really love the idea of a baby friendly (soft) photo album - which offers not only a glimpse of their favorite sight, but a chance to see significant faces that they may or may not encounter each day (such as grandparents and other extended family). There are tons of them on the market. Look for something safe and soft.. and stock it only with duplicate photos you’ll be ok with sustaining the kind of wear and tear that babies are so skilled at providing. This lovely video was shared to IG by thesabrinatan.show more