Загрузка видео...

Не удалось загрузить видео

На главную

Cache has an unlikely king Dottie, a Swedish pro for Entropy, currently tops the map. On Cache, he recently posted a 2.40 HLTV rating (23-14) vs. Mai Tai. He's quietly become one to watch as of late, averaging a 1.27 rating over his last three events despite poor team results.

21,358 просмотров • 2 месяцев назад •via X (Twitter)

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Day 11/90 of Inference Engineering How does vLLM work and how is it used in production? Before we discuss how vLLM works internally, it helps to understand what vLLM is. At a high level, vLLM is an inference engine that is designed to serve LLMs to thousands of concurrent users efficiently while managing scarce compute and memory. The goal for vLLM is to maximize throughput and minimize latency; optimizing for the best inference economics and experience for end users. With every request from the end user, it eventually ends up in the engine core, gets scheduled alongside other requests from other concurrent users, executes on the GPU, and updates the KV cache with the new key and value vectors, and streams the tokens back to the user. The Scheduler decides what requests should execute next while continuously batching requests together to maximize GPU utilization. Continuous batching is an inference optimization that allows new requests to join a running batch as other requests finish generating tokens. This helps with keeping the GPU utilization high instead of letting it sit idle waiting for an entire batch to complete generating. After the scheduler dispatches the selected batch to the Model Executor, the Model Executor prepares the tensors and metadata required for inference, retrieves each request’s block table from KV Cache Manager, launches the optimized transformer forward pass on the GPU, computes the logits, updates the KV cache with the new key and value vectors, and finally returns the results for sampling and streaming. The KV Cache Manager uses the PagedAttention memory layout to allocate fixed-size cache blocks on demand and maintains a Free Block Queue on the CPU that tracks which blocks in the GPU’s Paged KV Cache are currently free. When a request needs additional KV cache space, the KV Cache manager takes a free block from the queue and assigns it to that request, thus avoiding an expensive search through GPU memory for available cache blocks. All of these components form the core of vLLM’s inference engine. The Scheduler determines what requests are executed, the Model Executor determines how those requests are executed, the KV Cache Manager determines where each request’s KV cache lives using the PagedAttention Memory Layout. This architecture enables vLLM to serve thousands of concurrent requests with high throughput, low latency, and efficient GPU memory utilization. Heres a little animation that visualizes everything! - I've also completed the forward pass for my mnist.c project. I had a nice chat with shrey birmiwal, such a knowledgeable guy. Excited to learn more about vLLM and implement a tiny-vLLM one day.

max fu

70,797 просмотров • 2 месяцев назад

Shifting focus to 2025, RHP Tyler Bremner (UC Santa Barbara Baseball) is one of my favorite College arms in the class. As a freshman, Bremner split time between the bullpen and the rotation and pitched his way to a modest 5.37 ERA with an impressive strikeout-to-walk ratio of 80-to-17 across 55.1 innings. Bremner has a lean, athletic build at 6'2" and 180-lbs. He has room to fill out physically and as he does, his already-electric stuff will only tick up. Bremner has a low maintenance, under control operation with present arm speed. He attacks hitters from a high-three quarter slot and his fastball sits in the 93-96 range and tops out at 98. It has elite riding life (averaged over 21" of IVB in '23) and is most effective when located in the top-half of the zone. Between its velocity and shape, it is a 60-grade offering. He boasts two quality secondary offerings in a mid-80s changeup and low-80s slider. Bremner’s changeup is the more polished of the two, but his sweeping slider has plus potential. In 2023, Bremner’s changeup generated an impressive 48% miss rate. He has advanced feel for the offering, and he will throw it to both left and righthanded hitters. It plays well off his mid-90s fastball and flashes big time late tumbling life and some fade to the arm side. Bremner threw his slider just 12% of the time in 2023, but it has the chance to be a true out pitch in the future. It features ample late sweeping life as well as some depth, and is a real weapon especially against righthanded hitters. It has reportedly taken a step forward this fall and could be a lethal offering in 2024. Between his plus control, low effort delivery, and the chance to have three plus or better pitches, Bremner profiles as a starter professionally. He has a chance to become a household name this spring and with two strong seasons pitching in the rotation, Bremner has 1st Round upside in 2025. (📽️: UC Santa Barbara Baseball)

Peter Flaherty III

47,382 просмотров • 2 лет назад

RHP/1B Trace Phillips (Middle Tennessee Baseball) is an interesting draft-eligible sophomore on which to keep close tabs. Last spring, he hit .304/.360/.547 with 8 2B, 13 HR and 47 RBIs, but it’s his ability on the mound that's the root of his buzz. Phillips' back of the baseball card numbers last season are modest, but there is more to it than meets the eye. At 6'3" and 185-lbs, Phillips has an athletic, high-waisted frame. In the box, he has a simple setup with an ear-high handset and the bat horizontal above his back shoulder. Phillips has a slight load in which he has a noticeable barrel tip and a small stride. He moves well in the box and has some scissor action with his back leg. Phillips’ pure hit tool is fringey, but he has plus raw power and has shown the ability to drive the baseball to all fields. Max EV last spring of 109.4, if he's able to lift the ball more it will lead to a more impressive power output. Phillips’ current prospect status is largely driven by his ability on the mound. His numbers were unimpressive last season, but he took a step forward this fall. There’s some low hanging fruit to clean up in his delivery, but he has a short, whippy arm stroke and attacks from a low-three quarters slot with plenty of arm speed. Phillips’ FB has been up to 96 with ride through the zone (particular life at the top), though his best pitch is a high-70s-to-low-80s CH. He does a nice job of killing spin on it, and It averaged almost 15 MPH off his fastball this fall. It’s a plus pitch that flashes both fade to the arm side as well late tumble. Phillips completes his arsenal with a mid-80s slider that flashed above-average this fall with sharp, two-plane tilt. Both his changeup and slider profile as legitimate swing-and-miss offerings. Another positive development with Phillips this fall is that his strike-throwing took a step in the right direction. Mid-4th-6th round type this July. (📽️: Middle Tennessee Baseball)

Peter Flaherty III

10,984 просмотров • 1 год назад

While he won't toe the rubber this spring for the Hoosiers, RHP Luke Sinnard (Indiana Baseball) is an arm to be familiar with come July. Began his college career at Western Kentucky, but blossomed last year as a Sophomore at Indiana and pitched his way to a 4.27 ERA with 114 Ks (led the Big Ten) to just 25 BB across 86.1 IP. Earned a Second Team All-Big Ten selection. Towering, workhorse frame at 6'8" and 230-lbs. Sinnard attacks from an ultra high-three quarter slot and possesses an outlier release height north of 7-feet. Somewhat of a deep pull here, but his arm slot is a little reminiscent of former College of Charleston RHP William Privette, though not as sling-shotty or true OTT. His FB sits in the 91-94 range (T96) but it plays up thanks to the carry Sinnard is able to get on it through the zone. Last year, it averaged over 2,500 RPMs and over 19" of IVB. Sinnard's low-80s CB is a true bat-misser. Flashes immense depth and almost true 12-to-6 shape with some teeth. He has good feel for the pitch and it is effective against both right and lefthanded hitters. Held opponents to a .118 average and generated a 46% miss rate last spring. Also features an effective high-80s Cut/SL hybrid. Boasts late lateral life with a touch of depth. Gets under the hands of LHH. Another pitch that Sinnard has plus control of. Lastly, Sinnard throws a low-80s Split/CH. Feel for it is inconsistent, but it gets good separation off the FB and at times flashed tumbling life. It's an okay 4th offering. Sinnard has pro-level starter written all over him and even though you won't see him this season, he profiles as a potential 4th-7th round pick in this year's Draft. (📽️: Indiana Baseball)

Peter Flaherty III

36,630 просмотров • 2 лет назад