Загрузка видео...

Не удалось загрузить видео

На главную

We heavily optimized our GLTF files to fit any GPU memory. We built a custom Electron app to reduce polycount & convert textures to GPU-friendly Basis/ETC1S. Compressed textures used ~10-100x less memory & uploaded instantly, preventing decoding lags while scrolling!

17,465 просмотров • 8 месяцев назад •via X (Twitter)

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Dylan Patel on the importance of memory and storage Two key quotes: "An $NVDA GPU is faster than an $AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads." “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly" Full Quote: “We have over $80 million of compute: GPUs from $NVDA and $AMD, TPUs from Google, and Trainium from Amazon. We constantly run this benchmark using the newest inference engines, drivers, PyTorch versions, and other software. It runs every day through automated CI across the latest Chinese models from GLM, Zhipu, Moonshot, Kimi, Alibaba, and others. Initially, when we were benchmarking the differences between these chips, inference engines, and parallelism schemes, we used fixed context lengths. But with Agent X, we have now analyzed more than $5 million worth of Claude Code traces. This is real production traffic that users have donated to us, combined with internally generated data, so we now understand what an actual agent workload looks like. When we implement those workloads and run the benchmarks, it turns out that the chip you are using is very important, but how you handle memory offload can be even more important. An Nvidia GPU is faster than an AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads. Similarly, you can use a less powerful GPU with a much better storage solution and outperform the best GPU when it lacks those solutions. Simply buying the newest GPU does not necessarily give you the best inference economics. You need to layer in other innovations, including storage and memory.” Interviewer: “Who is the top player on your chart? Can you tell us?” Dylan Patel: “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly.”

Daniel Romero

38,220 просмотров • 1 месяц назад

researchers gave a tiny local model human-style memory and its context limit basically stopped existing a team from MBZUAI, Princeton and Weizmann took a 1B model and rebuilt how it reads. instead of attending to everything at once, the model reads in 1,024 token chunks and passes the important stuff forward through an associative memory, the same way you carry the plot of a book between chapters without rereading them. the design mirrors human memory on purpose. full attention inside a chunk works as short-term memory. the module that carries information between chunks works as long-term memory. they even trained it like a person, starting with short easy texts and raising the difficulty gradually, because memory thrown into the deep end learns nothing. the numbers back it up. the normal model burns 40GB of GPU memory on a long document and collapses hard past its limit, dropping from 0.86 to 0.32 accuracy. the memory version holds 0.71 at double that length while using a flat 12GB no matter how long the input gets. it also needs about 30% fewer FLOPs. the part i keep thinking about is that nobody scaled anything here. they didn't build a bigger model, didn't stretch the window, didn't add compute. they looked at how a brain handles a long day and copied the architecture. a model small enough to run on a consumer gpu now survives documents its own architecture used to choke on. we keep treating intelligence as a compute problem. sometimes it's a memory problem.

Alex Veremeyenko

16,147 просмотров • 1 месяц назад

Etched is deploying two new technologies in chip design: low-voltage inference and cluster-scale memory. CEO Gavin Uberti says they'll make their chips much more power-efficient and way, way faster than today's leading GPUs. He breaks it down: "We looked at a lot of early research directions, and we realized the key things that models need are way more compute and way faster memory." "If you think about inference, there are two key parts: prefill and decode. For prefill, it's a compute-bound problem. You need to have more FLOPS, more operations per second on each of your chips." "On our GPU, the bottleneck's actually thermals. You can't really run a GPU at more than around 50% of what it could theoretically do, or it'll melt." "So we're using a new technology today called low-voltage inference to try to solve this problem. You bring the voltage of the chip down dramatically, which allows us to have way, way better efficiency in terms of how much power is drawn per unit of math, and thus fit way way more flops onto the chip..." "For decode, it's all about bandwidth. Not just bandwidth on a chip, but bandwidth across your cluster. That's why we have this technology we call cluster-scale memory. It reduces the amount of time it takes to communicate from one chip to another dramatically." "As a result we can go use all of our HBM, HBM bandwidth, SRAM, SRAM bandwidth, and our scale-up domain as a single coherent pool. And that means if you're a user, you can go get much faster tokens per second, while still keeping your costs low."

TBPN

20,404 просмотров • 1 месяц назад

I gave a talk at GPU MODE workshop last week on llm.c - the origin story of llm.c - being naked in the world without PyTorch and having to re-invent Array, Autograd, Device, Dtype, Compile, Distributed - how to port a PyTorch layer to 1) explicit PyTorch - and then to 2) write the backward pass - 3) port forward & backward pass to C - 4) string all the layers together - achieving one file of C with no dependencies that compiles and runs ~instantly, where all memory is pre-planned and allocated a single time, fully deterministic, portable code that can run on a potato or a von Neumann probe - how most of llm.c was built at 1am-7am in a water villa porch in Maldives and why this is the recommended way to develop software - convert all of it to run in CUDA on GPU in fp32 - port matmul to cuBLAS - port attention to cuDNN flash-attention - introduce bfloat16 mixed precision - introduce many more optimizations and features like kernel fusions, Packed128, stochastic rounding, full determinism - add multi-GPU training, NCCL, sharded optimizer - add multi-node with MPI or file system or socket - reproduce GPT-2 (1.6B) on one 8XH100 node in 24 hours for $672 in llm.c, achieving (at the time) 29% less memory, 19% faster training that PyTorch nightly, and much faster compile & run - how open source development attracts Avengers from the internet - port to training Llama 3 imminent (branch exists) - many other notable forks - last thought: how software abstractions like Python/PyTorch and everything else really exist only because humans are finite in knowledge, IQ and attention, and how with increasing AI capability LLMs may export custom binaries like llm.c for any application directly, tearing apart and refactoring all abstractions as needed. More links in reply

Andrej Karpathy

337,263 просмотров • 1 год назад

Ghostty is getting automatic scrollback compression, resulting in 70 to 90% less physical memory usage. It happens incrementally when idle, so it had no measurable effect on IO throughput. I'm not aware of any other mainstream terminal that does this. Demo video below! The gains let us increase the default scrollback limit from 10MB to 50MB, because on average a full scrollback will still compress smaller than the prior limit. More history, for free. ("Unlimited", disk-paged history is on the roadmap too) Let's talk about cool implementation details, cause this was fun. First, the data structure and memory layout ("PageList") I wrote two years ago finally pays off! One of its traits is that screen memory is backed by a linked list of page-aligned, page-sized (or page-multiple-sized) blocks. Because each block is page-aligned and page-sized, we can use madvise to discard its physical backing while keeping the virtual address space reserved. Compressed pages therefore disappear from resident memory, but decompression is still guaranteed because the address space remains valid and we simply fault new pages back in as needed. We use the same trick for our memory pools, too. Unallocated pool pages don't count as resident memory, saving another couple of MB per terminal. This functionality is also available to libghostty-vt consumers via new `ghostty_terminal_compress` APIs. The consumer decides when the appropriate time to compress is and the APIs advise on compressability.

Mitchell Hashimoto

166,748 просмотров • 1 месяц назад

Stratosphere was our biggest and heaviest character on #Transformers ROTB. He was a real challenge to deal with on the Modeling, Texturing, and Rendering side. He had 1434 UDIMs due to his immense scale and was made up of tens of thousands of objects. His level of detail was truly something else. His vehicle form was also a behemoth. We had also built a mortar for his robot form and a cannon for his vehicle form but neither of these made it into the film as his part in the final battle was cut out. He was originally supposed to help out in the final battle, using both of these weapons against the Sweepers and Predacons. The decision to cut him from the final act came very late as all of his assets were final at that point. We were never told the reason why his role was cut from the film. I had to re-design my texturing system in Mari when working on him to ensure that it could handle such a complex and large character. I had to simplify a lot of the procedural systems and even had to branch off some of them into their own files to ensure that artists could work with somewhat decent performance. We ran into a lot of issues pushing him through our pipeline, both as a model as well as rendering him due to the sheer amount of objects and textures that had to be processed. For shots, it was requested by Lighting that we reduce his memory footprint as Google had complained he was taking up too much memory and their I/O for rendering on the cloud was being affected. The leadership team and I decided that the best way to do this was to half the texel density on the parts that wouldn't be visible in our approved shots and to also half the resolution of the textures in the parts not visible to camera. This allowed us to optimize his memory footprint both in shots as well as storage and I/O. It was massive technical undertaking working on Stratosphere, so much so that we had to delay getting him into shots as we simply couldn't push him through with our normal tooling. I really enjoyed the challenge that he posed for us as a team. CREDITS: Primary Modeling by Oscar Lowe Support Modeling by Arthur Grandjean and his team Final Texturing & Lookdev by Yaz Raji Video credits: Breakdown shot by MPC Film footage by Paramount

Rassoul Edji

268,448 просмотров • 1 год назад

The #GTA 3 port to the Sega #Dreamcast has been progressing at an incredible pace. It's been amazing to see the whole DC community come together to tag-team this "impossible" project... Here it is running on a stock DC, no longer requiring the 32MB RAM hardware modification, only a few weeks into development. Since I finally got the time to sit down, build the codebase, and look into some of what I think is the critical path for performance, let's talk about some technical shit, and some of the future steps I think can be taken to further improve performance. First of all, I want everyone to take note that this is NOT a port of the PS2 version. This is a port of the PC version, which has extra content, increased draw distance, improved textures, and other things that have actually increased the challenge here... Whether the DC version will ultimately have these additions or not will remain to be seen, but we're running into plenty of shit that the PS2 didn't have to worry about (like these big-ass PC replay saves won't fit onto a Visual Memory Unit!) Secondly, lets talk about what is and isn't currently optimized, because it's absolutely vital that the DC's hardware is fully utilized here for the sake of performance and achieving a competitive polygon count. Unlike with modern devices, where the whole graphics pipeline is handled by the GPU, both the PS2 and Dreamcast were responsible for transforming and doing lighting calculations for each vertex BEFORE they got submitted to the GPU. The PS2 had a vector coprocessor to do this, while the Dreamcast had a few extremely important SIMD and fast math assembly instructions on its CPU to do these computations. Up until literally just a few hours ago (not shown in this footage), the Dreamcast's SH4 was doing 100% of these operations in slow-ass plain C and C++ code, which is absolutely sub-optimal and is immediately bogging down its CPU with just transforming vertices, bottlenecking the entire graphics pipeline on the fist T&L stage, and also leaving less CPU time for handling other gameplay logic... this is going to absolutely have to be addressed (and already has begun to be). Another issue that is crippling performance here is the fact that the models are all using individual triangles rather than triangle strips, which the Dreamcast's PVR GPU was designed to handle better... Converting these models to use strips rather than individual triangles will result in MANY different gainz for the DC, as you're going from 3N to N+2 vertices per triangle. Converting the models to triangle strips will 1) reduce load times, since model assets will be smaller 2) reduce the amount of video memory required to hold these vertices on the GPU 3) reduce the amount of shit that must be transferred from the CPU to the GPU and 4) give us back a bunch of CPU time, since the SH4 will be less bogged down transforming redundant vertices! TL;DR: This is still EXTREMELY suboptimal in terms of fully utilizing the graphical potential of the Dreamcast. There is going to be a LOT that can be done still to both improve performance and polygon counts, so stay tuned! FINALLY: Mad respect and love to Stefanos Kornilios Mitsis Poiitidis, for doing an amazing job leading this project, and to Frogbull , Esppiral, and everyone else who is helping us stick it to the PS2 by making this happen! #gamedev #retrogaming #cplusplus

Falco Girgis

88,356 просмотров • 2 лет назад