Загрузка видео...

Не удалось загрузить видео

На главную

pip install spectralquant ✂️ Up to 6.62x KV cache compression for LLMs and transformers. Same model. Faster outputs. Smaller KV cache. Try now (2 mins): - KV cache integration via Hugging Face's DynamicCache - Three presets: 5.95x (paper), 6.55x (validated), 6.68x (edge) - Mistral 7B / Qwen 2.5 7B...

anirudhbv_ce's profile picture

ani

6,917 subscribers

17,269 просмотров • 2 месяцев назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

🚀 Self-speculation brings 6.75x real speedup for LLM generation with SGLang inference! Same model drafts future tokens in Diffusion mode → then verifies them in AR (causal) mode. One model and one KV cache. Just different attention masks. Thanks to perfect alignment, we get 2× longer acceptance lengths than MTP techniques (Eagle-3, MTP, dFlash). We run 2 forward passes… but the 2× higher acceptance means we break even - and with zero overhead from extra drafter, KV cache, or LM head that comes with MTP - those are not free. Last week we released Nemotron-Labs-Diffusion + Tri-mode LLMs! We did continued pre-training on Ministral-3 models by switching attention patterns (block causal bidirectional). Result: one model that runs AR mode, Diffusion mode, and Self-Speculation. Diffusion mode already shows high benchmark accuracy - excited to see what happens when someone beats left-to-right acceptance! 🔥 Github: Paper: SGLang inference: Try the models on HF:
0:13

🚀 Self-speculation brings 6.75x real speedup for LLM generation with SGLang inference! Same model drafts future tokens in Diffusion mode → then verifies them in AR (causal) mode. One model and one KV cache. Just different attention masks. Thanks to perfect alignment, we get 2× longer acceptance lengths than MTP techniques (Eagle-3, MTP, dFlash). We run 2 forward passes… but the 2× higher acceptance means we break even - and with zero overhead from extra drafter, KV cache, or LM head that comes with MTP - those are not free. Last week we released Nemotron-Labs-Diffusion + Tri-mode LLMs! We did continued pre-training on Ministral-3 models by switching attention patterns (block causal bidirectional). Result: one model that runs AR mode, Diffusion mode, and Self-Speculation. Diffusion mode already shows high benchmark accuracy - excited to see what happens when someone beats left-to-right acceptance! 🔥 Github: Paper: SGLang inference: Try the models on HF:

Pavlo Molchanov

66,604 просмотров • 2 месяцев назад

#WATCH | Delhi | An AAP worker says, "Dictatorship will not be tolerated. You are picking up students as if they were terrorists. Yet, those responsible for leaking the papers are not being picked up. The government is failing to apprehend the paper leak culprits."
0:10

#WATCH | Delhi | An AAP worker says, "Dictatorship will not be tolerated. You are picking up students as if they were terrorists. Yet, those responsible for leaking the papers are not being picked up. The government is failing to apprehend the paper leak culprits."

ANI

24,605 просмотров • 2 месяцев назад

#WATCH | Delhi: Lok Sabha LoP and Congress MP Rahul Gandhi, along with INDIA bloc MPs, reach Gandhi Smriti as they continue their protest over the NEET paper leak.
0:18

#WATCH | Delhi: Lok Sabha LoP and Congress MP Rahul Gandhi, along with INDIA bloc MPs, reach Gandhi Smriti as they continue their protest over the NEET paper leak.

ANI

102,587 просмотров • 14 дней назад

#WATCH | Muskan from Chandigarh says, "We are here to support an Army. They are doing so much for us, and we also want to do something for our Army."
0:17

#WATCH | Muskan from Chandigarh says, "We are here to support an Army. They are doing so much for us, and we also want to do something for our Army."

ANI

576,317 просмотров • 1 год назад

#WATCH | TVK Chief Vijay meets Tamil Nadu Governor Rajendra Vishwanath Arlekar at the Lok Bhavan. With the support of Congress, CPI, CPI(M), VCK and IUML, 121 MLAs are now in support of TVK, and its path to form a government in the state is now clear. (Source: Lok Bhavan)
0:16

#WATCH | TVK Chief Vijay meets Tamil Nadu Governor Rajendra Vishwanath Arlekar at the Lok Bhavan. With the support of Congress, CPI, CPI(M), VCK and IUML, 121 MLAs are now in support of TVK, and its path to form a government in the state is now clear. (Source: Lok Bhavan)

ANI

234,400 просмотров • 2 месяцев назад

#WATCH | Patna | On tweet by Kerala unit of Congress, ' 'B for Beedi and B for Bihar...' (which stands deleted now), RJD leader Tejashwi Yadav says, "It was a wrong tweet. We don't support it."
0:11

#WATCH | Patna | On tweet by Kerala unit of Congress, ' 'B for Beedi and B for Bihar...' (which stands deleted now), RJD leader Tejashwi Yadav says, "It was a wrong tweet. We don't support it."

ANI

241,547 просмотров • 11 месяцев назад

#WATCH | Delhi: BJP MPs staged a protest outside the Parliament demanding the resignation of Punjab Education Minister Harjot Singh Bains, over the alleged paper leak in the state.
0:19

#WATCH | Delhi: BJP MPs staged a protest outside the Parliament demanding the resignation of Punjab Education Minister Harjot Singh Bains, over the alleged paper leak in the state.

ANI

63,386 просмотров • 10 дней назад

#WATCH | Delhi | BJP leader Smriti Irani says, "...Vikram talks about India's capacity to be the bridge not only between the global north and the global south, but also to ascend to a new formidable future. My compliments to the author and my gratitude for his service"
0:16

#WATCH | Delhi | BJP leader Smriti Irani says, "...Vikram talks about India's capacity to be the bridge not only between the global north and the global south, but also to ascend to a new formidable future. My compliments to the author and my gratitude for his service"

ANI

20,246 просмотров • 9 месяцев назад

#WATCH | Kolkata: Former CM and TMC chief Mamata Banerjee says, “If need be, I myself will go to the CJP protest. They had our support from the beginning, and will continue to have it.” (Source: TMC)
0:18

#WATCH | Kolkata: Former CM and TMC chief Mamata Banerjee says, “If need be, I myself will go to the CJP protest. They had our support from the beginning, and will continue to have it.” (Source: TMC)

ANI

124,861 просмотров • 16 дней назад

#WATCH | Auckland, New Zealand | Prime Minister Narendra Modi emplanes for India after concluding his three-nation visit to Indonesia, Australia, and New Zealand. (Source: ANI/DD)
0:18

#WATCH | Auckland, New Zealand | Prime Minister Narendra Modi emplanes for India after concluding his three-nation visit to Indonesia, Australia, and New Zealand. (Source: ANI/DD)

ANI

88,183 просмотров • 26 дней назад

#WATCH | On Bagram Airbase in Afghanistan, US President Donald J Trump says, "We're talking now to Afghanistan, and we want it back, and we want it back soon. If they don't do it, you're going to find out what I'm going to do." (Source: US Network Pool via Reuters)
0:19

#WATCH | On Bagram Airbase in Afghanistan, US President Donald J Trump says, "We're talking now to Afghanistan, and we want it back, and we want it back soon. If they don't do it, you're going to find out what I'm going to do." (Source: US Network Pool via Reuters)

ANI

461,828 просмотров • 10 месяцев назад

#WATCH | Guwahati, Assam: An FIR has been filed against the Assam Jatiya Parishad (AJP) candidate of the Guwahati Central seat, Kunki Chowdhury, at the Panbazar Police Station for violating the Model Code of Conduct on the polling day, April 9th. She says, "An FIR was filed against me, making some false allegations. So I came here to give a statement... I presented my side and gave my statement. I have full faith in the system, and it will investigate thoroughly..."
0:18

#WATCH | Guwahati, Assam: An FIR has been filed against the Assam Jatiya Parishad (AJP) candidate of the Guwahati Central seat, Kunki Chowdhury, at the Panbazar Police Station for violating the Model Code of Conduct on the polling day, April 9th. She says, "An FIR was filed against me, making some false allegations. So I came here to give a statement... I presented my side and gave my statement. I have full faith in the system, and it will investigate thoroughly..."

ANI

28,172 просмотров • 3 месяцев назад

#WATCH | Rudraprayag, Uttarakhand | Due to fresh snowfall in April, the Kedarnath temple complex and its surrounding areas are currently draped in a pristine white blanket of snow. Along the trekking route, glaciers have descended near Chhoti Lincholi and Badi Lincholi. As pilgrims advance from Gaurikund toward Lincholi, they will traverse paths carved directly through these glaciers, adding a thrilling edge to their journey. Furthermore, the view of Baba Kedarnath Dham from Dev Darshani is exceptionally divine and captivating this season, promising a deeply spiritual experience for all visitors. The portals (kapat) of Baba Kedarnath are set to open for devotees on April 22. With the opening of the gates, thousands of pilgrims are expected to arrive at the Dham, marking the formal commencement of this holy pilgrimage. (Source: Local Administration)
0:07

#WATCH | Rudraprayag, Uttarakhand | Due to fresh snowfall in April, the Kedarnath temple complex and its surrounding areas are currently draped in a pristine white blanket of snow. Along the trekking route, glaciers have descended near Chhoti Lincholi and Badi Lincholi. As pilgrims advance from Gaurikund toward Lincholi, they will traverse paths carved directly through these glaciers, adding a thrilling edge to their journey. Furthermore, the view of Baba Kedarnath Dham from Dev Darshani is exceptionally divine and captivating this season, promising a deeply spiritual experience for all visitors. The portals (kapat) of Baba Kedarnath are set to open for devotees on April 22. With the opening of the gates, thousands of pilgrims are expected to arrive at the Dham, marking the formal commencement of this holy pilgrimage. (Source: Local Administration)

ANI

15,161 просмотров • 3 месяцев назад

#WATCH | Andhra Pradesh Deputy CM Pawan Kalyan arrives in Delhi to participate in tomorrow’s “Sena Prasthanam… For National Integrity” programme. During the programme, Pawan Kalyan is expected to give direction and guidance to party leaders on the movement’s vision and future course of action. (Source: Janasena party office)
0:13

#WATCH | Andhra Pradesh Deputy CM Pawan Kalyan arrives in Delhi to participate in tomorrow’s “Sena Prasthanam… For National Integrity” programme. During the programme, Pawan Kalyan is expected to give direction and guidance to party leaders on the movement’s vision and future course of action. (Source: Janasena party office)

ANI

34,005 просмотров • 1 месяц назад

#WATCH | Responding to ANI's question on the future of India-US trade negotiations in view of US sanctions imposed on India, White House economic adviser Kevin Hassett says, "The trade team and the President are disappointed that India continues to fund Russia's Ukraine war. Hopefully, it's a diplomatic issue that will have positive developments soon." (Source: US Network Pool via Reuters)
0:15

#WATCH | Responding to ANI's question on the future of India-US trade negotiations in view of US sanctions imposed on India, White House economic adviser Kevin Hassett says, "The trade team and the President are disappointed that India continues to fund Russia's Ukraine war. Hopefully, it's a diplomatic issue that will have positive developments soon." (Source: US Network Pool via Reuters)

ANI

82,536 просмотров • 11 месяцев назад

#WATCH | Haryana: A video went viral where a man was throwing currency notes from his running car in Gurugram. Police file a case in the matter. (Police have verified the viral video)
0:15

#WATCH | Haryana: A video went viral where a man was throwing currency notes from his running car in Gurugram. Police file a case in the matter. (Police have verified the viral video)

ANI

895,111 просмотров • 3 лет назад

#WATCH | Hyderabad, Telangana | After the meeting with SP chief Akhilesh Yadav, BRS working president KT Rama Rao says, "We draw inspiration from you and your party about the manner in which you reached Delhi with 37 MPs. We will bounce back once again and try to come back to power..."
0:18

#WATCH | Hyderabad, Telangana | After the meeting with SP chief Akhilesh Yadav, BRS working president KT Rama Rao says, "We draw inspiration from you and your party about the manner in which you reached Delhi with 37 MPs. We will bounce back once again and try to come back to power..."

ANI

209,095 просмотров • 7 месяцев назад

#WATCH | Defence Minister Rajnath Singh leaves from 7, LKM. Prime Minister Narendra Modi held a meeting with Defence Minister Rajnath Singh, NSA Ajit Doval, CDS and Chiefs of all three Services, at 7, LKM.
0:19

#WATCH | Defence Minister Rajnath Singh leaves from 7, LKM. Prime Minister Narendra Modi held a meeting with Defence Minister Rajnath Singh, NSA Ajit Doval, CDS and Chiefs of all three Services, at 7, LKM.

ANI

197,226 просмотров • 1 год назад