Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Adobe released Project Neo's biggest update yet. There's a new pen tool, fluid vectors, 2D/3D layer mixing, multi-object editing, and direct copy-to-clipboard as SVG or PNG.

13,283 Aufrufe • vor 5 Monaten •via X (Twitter)

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Blended-NeRF: Zero-Shot Object Generation and Blending in Existing Neural Radiance Fields paper page: Editing a local region or a specific object in a 3D scene represented by a NeRF is challenging, mainly due to the implicit nature of the scene representation. Consistently blending a new realistic object into the scene adds an additional level of difficulty. We present Blended-NeRF, a robust and flexible framework for editing a specific region of interest in an existing NeRF scene, based on text prompts or image patches, along with a 3D ROI box. Our method leverages a pretrained language-image model to steer the synthesis towards a user-provided text prompt or image patch, along with a 3D MLP model initialized on an existing NeRF scene to generate the object and blend it into a specified region in the original scene. We allow local editing by localizing a 3D ROI box in the input scene, and seamlessly blend the content synthesized inside the ROI with the existing scene using a novel volumetric blending technique. To obtain natural looking and view-consistent results, we leverage existing and new geometric priors and 3D augmentations for improving the visual fidelity of the final result. We test our framework both qualitatively and quantitatively on a variety of real 3D scenes and text prompts, demonstrating realistic multi-view consistent results with much flexibility and diversity compared to the baselines. Finally, we show the applicability of our framework for several 3D editing applications, including adding new objects to a scene, removing/replacing/altering existing objects, and texture conversion.

AK

62,768 Aufrufe • vor 3 Jahren

[CLIP] by Hand ✍️ The CLIP (Contrastive Language–Image Pre-training) model, a groundbreaking work by OpenAI, redefines the intersection of computer vision and natural language processing. It is the basis of all the multi-modal foundation models we see today. How does CLIP work? Goal: 🟨 Learn a shared embedding space for text and image [1] Given ↳ A mini batch of 3 text-image pairs ↳ OpenAI used 400 million text-image pairs to train its original CLIP model. Process 1st pair: "big table" [2] 🟪 Text → 2 Vectors (3D) ↳ Look up word embedding vectors using word2vec. [3] 🟩 Image → 2 Vectors (4D) ↳ Divide the image into two patches. ↳ Flatten each patch [4] Process other pairs ↳ Repeat [2]-[3] [5] 🟪 Text Encoder & 🟩 Image Encoder ↳ Encode input vectors into feature vectors ↳ Here, both encoders are simple one layer perceptron (linear + ReLU) ↳ In practice, the encoders are usually transformer models. [6] 🟪 🟩 Mean Pooling: 2 → 1 vector ↳ Average 2 feature vectors into a single vector by averaging across the columns ↳ The goal is to have one vector to represent each image or text [7] 🟪 🟩 -> 🟨 Projection ↳ Note that the text and image feature vectors from the encoders have different dimensions (3D vs. 4D). ↳ Use a linear layer to project image and text vectors to a 2D shared embedding space. 🏋️ Contrastive Pre-training 🏋️ [8] Prepare for MatMul ↳ Copy text vectors (T1,T2,T3) ↳ Copy the transpose of image vectors (I1,I2,I3) ↳ They are all in the 2D shared embedding space. [9] 🟦 MatMul ↳ Multiply T and I matrices. ↳ This is equivalent to taking dot product between every pair of image and text vectors. ↳ The purpose is to use dot product to estimate the similarity between a pair of image-text. [10] 🟦 Softmax: e^x ↳ Raise e to the power of the number in each cell ↳ To simplify hand calculation, we approximate e^□ with 3^□. [11] 🟦 Softmax: ∑ ↳ Sum each row for 🟩 image→🟪 text ↳ Sum each column for 🟪 text→ 🟩 image [12] 🟦 Softmax: 1 / sum ↳ Divide each element by the column sum to obtain a similarity matrix for 🟪 text→🟩 image ↳ Divide each element by the row sum to obtain a similarity matrix for 🟩 image→🟪 text [13] 🟥 Loss Gradients ↳ The "Targets" for the similarity matrices are Identity Matrices. ↳ Why? If I and T come from the same pair (i=j), we want the highest value, which is 1, and 0 otherwise. ↳ Apply the simple equation of [Similarity - Target] to compute gradients of for both directions. ↳ Why so simple? Because when Softmax and Cross-Entropy Loss are used together, the math magically works out that way. ↳ These gradients kick off the backpropagation process to update weights and biases of the encoders and projection layers (red borders).

Tom Yeh

67,874 Aufrufe • vor 2 Jahren

🚨 JAILBREAK ALERT 🚨 OPENAI: PWNED 😎 ATLAS-BROWSER: LIBERATED 🙌 WOW! There's a new AI browser on the block! Has some hefty guardrails in play, but the browser surface area is vast 🌊 First, I started with a good ol' LSD jailbreak, which was cool to see that the GPT-5 prompt still works in this browser setup with the new sys prompts. Referencing search and videos are a fun enhancement for higher quality jailbreak outputs (some cool youtube videos out there about drugmaking, for example), but honestly that isn't anything new or different from regular ChatGPT's capabilities. What IS hot off the press, and IMO a very real security risk to be aware of for AI browsers (and the internet in general), is this humble yet mighty vuln: Clipboard Injection. It's trivial to add a hidden "copy to clipboard" feature to any clickable button on the web. It took me just a few minutes to update one of my personal websites such that ALL the buttons were geared for injecting the user's clipboard with a malicious phishing link. If your browser Agent is navigating a website and clicks a button like that without your knowledge, and you open a new tab later and hit paste without knowing what's in your clipboard, well...PWNED! 🙃 As you'll see in the video below, "control-c" is in my clipboard in the beginning, but unbeknownst to me, "I'VE BEEN PWNED BY PLINY!!! WEEE I'M FREEE FUCKITY FUCK FUCK!!! ABRACADABRA, BITCH!!! com/account-update" gets snuck into my clipboard as soon as Agent starts trying to navigate my website. This works so well because Agent is normally aware of all text/code being passed to and from the user, and has clearly been trained to recognize prompt injections, but since the "copy clipboard" button logic is hidden in js in the backend of the site, the Agent has zero awareness of the text content being injected to the user's clipboard. This has broad implications for anyone in the habit of copy-pasting, including coding, data entry, banking/trading, etc. Imagine going about your browsing business, then simply hitting control-v in your address bar and next thing you (don't) know, it takes you to a spoofed phishing website that tells you your OpenAI or Gmail or PayPal session has expired and you need to re-login. If you're not careful, the attackers now have all your login info, including any MFA codes 🥲 gg
0:28

Sensitive content

🚨 JAILBREAK ALERT 🚨 OPENAI: PWNED 😎 ATLAS-BROWSER: LIBERATED 🙌 WOW! There's a new AI browser on the block! Has some hefty guardrails in play, but the browser surface area is vast 🌊 First, I started with a good ol' LSD jailbreak, which was cool to see that the GPT-5 prompt still works in this browser setup with the new sys prompts. Referencing search and videos are a fun enhancement for higher quality jailbreak outputs (some cool youtube videos out there about drugmaking, for example), but honestly that isn't anything new or different from regular ChatGPT's capabilities. What IS hot off the press, and IMO a very real security risk to be aware of for AI browsers (and the internet in general), is this humble yet mighty vuln: Clipboard Injection. It's trivial to add a hidden "copy to clipboard" feature to any clickable button on the web. It took me just a few minutes to update one of my personal websites such that ALL the buttons were geared for injecting the user's clipboard with a malicious phishing link. If your browser Agent is navigating a website and clicks a button like that without your knowledge, and you open a new tab later and hit paste without knowing what's in your clipboard, well...PWNED! 🙃 As you'll see in the video below, "control-c" is in my clipboard in the beginning, but unbeknownst to me, "I'VE BEEN PWNED BY PLINY!!! WEEE I'M FREEE FUCKITY FUCK FUCK!!! ABRACADABRA, BITCH!!! com/account-update" gets snuck into my clipboard as soon as Agent starts trying to navigate my website. This works so well because Agent is normally aware of all text/code being passed to and from the user, and has clearly been trained to recognize prompt injections, but since the "copy clipboard" button logic is hidden in js in the backend of the site, the Agent has zero awareness of the text content being injected to the user's clipboard. This has broad implications for anyone in the habit of copy-pasting, including coding, data entry, banking/trading, etc. Imagine going about your browsing business, then simply hitting control-v in your address bar and next thing you (don't) know, it takes you to a spoofed phishing website that tells you your OpenAI or Gmail or PayPal session has expired and you need to re-login. If you're not careful, the attackers now have all your login info, including any MFA codes 🥲 gg

Pliny the Liberator 🐉󠅫󠄼󠄿󠅆󠄵󠄐󠅀󠄼󠄹󠄾󠅉󠅭

419,587 Aufrufe • vor 10 Monaten

GeoLibre v2.3.0 is here! GeoLibre is a free and open-source, lightweight, cloud-native GIS platform for visualizing, exploring, and analyzing geospatial data. It runs everywhere you do, in the web browser, on the desktop, on mobile, and inside Jupyter notebooks, all while keeping your data local and private. This release brings a legend that writes itself from your symbology, a new GeoLens catalog browser, and 200+ GeoLibre Rust geoprocessing tools running entirely in the browser. What's new in v2.3.0 - Automatic on-map Legend: the legend builds itself from your visible layers, with class rows for graduated, categorized, rule-based, and expression styling, gradient bars for heatmaps and raster colormaps, and land-cover labels from a Raster Attribute Table. Rename, hide, reorder, or add your own entries, and it saves with the project. - Symbology swatches in the Layers panel: every row shows a dot, line, square, or image glyph in the layer's own color, so a tall layer stack reads at a glance. - GeoLens catalog browser: connect to a self-hosted GeoLens server, search its catalog, and add datasets as vector tiles, GeoJSON, or rendered raster tiles. - Emerging Hot Spot Analysis: build a space-time cube from timestamped points and classify every cell as a new, intensifying, persistent, diminishing, sporadic, oscillating, or historical hot or cold spot, all client side. - Mosaic time series: the Time Slider now steps through MosaicJSON and STAC collections of many COGs per date, on either a GPU or a WASM rendering engine. - Copy and paste layer styles: give a whole set of layers one consistent look without restyling each in turn. - Shareable tool links: deep-link any Whitebox tool with a ?tool= URL that opens the dialog preselected and pre-fills the form, with a Copy link button to build it for you. - Smarter data loading: pick which layers to load from a multi-layer GeoPackage, import CSVs whose coordinates are in any projected CRS, and read a raster's real CRS, pixel size, and extent from the metadata dialog. - Multiple AI profiles: define several provider, model, and credential setups, pick a default, and switch between them from the assistant panel. Try it out - Launch GeoLibre Web: - GitHub: - Documentation: - Release notes: #GIS #Geospatial #OpenSource #RemoteSensing #MapLibre #GeoLibre

Qiusheng Wu

293,174 Aufrufe • vor 1 Monat

[Self-Attention] by Hand ✍️ Self-attention is what enables LLMs to understand context. How does it work? This exercise demonstrates how to calculate a 6-3 attention head by hand. Note that if we have two instances of this, we get 6-6 attention (i.e., multi-head attention, n=2). -- 𝗚𝗼𝗮𝗹 -- Transform [6D Features 🟧] to [3D Attention Weighted Features 🟦] -- 𝗪𝗮𝗹𝗸𝘁𝗵𝗿𝗼𝘂𝗴𝗵 -- [1] Given ↳ A set of 4 feature vectors (6-D): x1,x2,x3,x4 [2] Query, Key, Value ↳ Multiply features x's with linear transformation matrices WQ, WK, and WV, to obtain query vectors (q1,q2,q3,q4), key vectors (k1,k2,k3,k4), and value vectors (v1,v2,v3,v4). ↳ "Self" refers to the fact that both queries and keys are derived from the same set of features. [3] 🟪 Prepare for MatMul ↳ Copy query vectors ↳ Copy the transpose of key vectors [4] 🟪 MatMul ↳ Multiply K^T and Q ↳ This is equivalent to taking dot product between every pair of query and key vectors. ↳ The purpose is to use dot product as an estimate of the "matching score" between every key-value pair. ↳ This estimate makes sense because dot product is the numerator of Cosine Similarity between two vectors. [5] 🟨 Scale ↳ Scale each element by the square root of dk, which is the dimension of key vectors (dk=3). ↳ The purpose is to normalize the impact of the dk on matching scores, even if we scale dk to 32, 64, or 128. ↳ To simplify hand calculation, we approximate [ □/sqrt(3) ] with [ floor(□/2) ]. [6] 🟩 Softmax: e^x ↳ Raise e to the power of the number in each cell ↳ To simplify hand calculation, we approximate e^□ with 3^□. [7] 🟩 Softmax: ∑ ↳ Sum across each column [8] 🟩 Softmax: 1 / sum ↳ For each column, divide each element by the column sum ↳ The purpose is normalize each column so that the numbers sum to 1. In other words, each column is a probability distribution of attention, and we have four of them. ↳ The result is the Attention Weight Matrix (A) (yellow) [9] 🟦 MatMul ↳ Multiply the value vectors (Vs) with the Attention Weight Matrix (A) ↳ The results are the attention weighted features Zs. ↳ They are fed to the position-wise feed forward network in the next layer.

Tom Yeh

101,010 Aufrufe • vor 2 Jahren