Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

pip install spectralquant ✂️ Up to 6.62x KV cache compression for LLMs and transformers. Same model. Faster outputs. Smaller KV cache. Try now (2 mins): - KV cache integration via Hugging Face's DynamicCache - Three presets: 5.95x (paper), 6.55x (validated), 6.68x (edge) - Mistral 7B / Qwen 2.5 7B...

anirudhbv_ce's profile picture

ani

6,793 subscribers

16,583 Aufrufe • vor 1 Monat •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

🚀 Self-speculation brings 6.75x real speedup for LLM generation with SGLang inference! Same model drafts future tokens in Diffusion mode → then verifies them in AR (causal) mode. One model and one KV cache. Just different attention masks. Thanks to perfect alignment, we get 2× longer acceptance lengths than MTP techniques (Eagle-3, MTP, dFlash). We run 2 forward passes… but the 2× higher acceptance means we break even - and with zero overhead from extra drafter, KV cache, or LM head that comes with MTP - those are not free. Last week we released Nemotron-Labs-Diffusion + Tri-mode LLMs! We did continued pre-training on Ministral-3 models by switching attention patterns (block causal bidirectional). Result: one model that runs AR mode, Diffusion mode, and Self-Speculation. Diffusion mode already shows high benchmark accuracy - excited to see what happens when someone beats left-to-right acceptance! 🔥 Github: Paper: SGLang inference: Try the models on HF:
0:13

🚀 Self-speculation brings 6.75x real speedup for LLM generation with SGLang inference! Same model drafts future tokens in Diffusion mode → then verifies them in AR (causal) mode. One model and one KV cache. Just different attention masks. Thanks to perfect alignment, we get 2× longer acceptance lengths than MTP techniques (Eagle-3, MTP, dFlash). We run 2 forward passes… but the 2× higher acceptance means we break even - and with zero overhead from extra drafter, KV cache, or LM head that comes with MTP - those are not free. Last week we released Nemotron-Labs-Diffusion + Tri-mode LLMs! We did continued pre-training on Ministral-3 models by switching attention patterns (block causal bidirectional). Result: one model that runs AR mode, Diffusion mode, and Self-Speculation. Diffusion mode already shows high benchmark accuracy - excited to see what happens when someone beats left-to-right acceptance! 🔥 Github: Paper: SGLang inference: Try the models on HF:

Pavlo Molchanov

66,519 Aufrufe • vor 1 Monat

#WATCH | Delhi | An AAP worker says, "Dictatorship will not be tolerated. You are picking up students as if they were terrorists. Yet, those responsible for leaking the papers are not being picked up. The government is failing to apprehend the paper leak culprits."
0:10

#WATCH | Delhi | An AAP worker says, "Dictatorship will not be tolerated. You are picking up students as if they were terrorists. Yet, those responsible for leaking the papers are not being picked up. The government is failing to apprehend the paper leak culprits."

ANI

24,605 Aufrufe • vor 2 Monaten

#WATCH | Muskan from Chandigarh says, "We are here to support an Army. They are doing so much for us, and we also want to do something for our Army."
0:17

#WATCH | Muskan from Chandigarh says, "We are here to support an Army. They are doing so much for us, and we also want to do something for our Army."

ANI

576,254 Aufrufe • vor 1 Jahr

#WATCH | TVK Chief Vijay meets Tamil Nadu Governor Rajendra Vishwanath Arlekar at the Lok Bhavan. With the support of Congress, CPI, CPI(M), VCK and IUML, 121 MLAs are now in support of TVK, and its path to form a government in the state is now clear. (Source: Lok Bhavan)
0:16

#WATCH | TVK Chief Vijay meets Tamil Nadu Governor Rajendra Vishwanath Arlekar at the Lok Bhavan. With the support of Congress, CPI, CPI(M), VCK and IUML, 121 MLAs are now in support of TVK, and its path to form a government in the state is now clear. (Source: Lok Bhavan)

ANI

234,332 Aufrufe • vor 2 Monaten

#WATCH | Patna | On tweet by Kerala unit of Congress, ' 'B for Beedi and B for Bihar...' (which stands deleted now), RJD leader Tejashwi Yadav says, "It was a wrong tweet. We don't support it."
0:11

#WATCH | Patna | On tweet by Kerala unit of Congress, ' 'B for Beedi and B for Bihar...' (which stands deleted now), RJD leader Tejashwi Yadav says, "It was a wrong tweet. We don't support it."

ANI

241,547 Aufrufe • vor 10 Monaten

#WATCH | Delhi | BJP leader Smriti Irani says, "...Vikram talks about India's capacity to be the bridge not only between the global north and the global south, but also to ascend to a new formidable future. My compliments to the author and my gratitude for his service"
0:16

#WATCH | Delhi | BJP leader Smriti Irani says, "...Vikram talks about India's capacity to be the bridge not only between the global north and the global south, but also to ascend to a new formidable future. My compliments to the author and my gratitude for his service"

ANI

20,246 Aufrufe • vor 8 Monaten

#WATCH | Auckland, New Zealand | Prime Minister Narendra Modi emplanes for India after concluding his three-nation visit to Indonesia, Australia, and New Zealand. (Source: ANI/DD)
0:18

#WATCH | Auckland, New Zealand | Prime Minister Narendra Modi emplanes for India after concluding his three-nation visit to Indonesia, Australia, and New Zealand. (Source: ANI/DD)

ANI

87,913 Aufrufe • vor 7 Tagen

#WATCH | On Bagram Airbase in Afghanistan, US President Donald J Trump says, "We're talking now to Afghanistan, and we want it back, and we want it back soon. If they don't do it, you're going to find out what I'm going to do." (Source: US Network Pool via Reuters)
0:19

#WATCH | On Bagram Airbase in Afghanistan, US President Donald J Trump says, "We're talking now to Afghanistan, and we want it back, and we want it back soon. If they don't do it, you're going to find out what I'm going to do." (Source: US Network Pool via Reuters)

ANI

461,828 Aufrufe • vor 10 Monaten

#WATCH | Rudraprayag, Uttarakhand | Due to fresh snowfall in April, the Kedarnath temple complex and its surrounding areas are currently draped in a pristine white blanket of snow. Along the trekking route, glaciers have descended near Chhoti Lincholi and Badi Lincholi. As pilgrims advance from Gaurikund toward Lincholi, they will traverse paths carved directly through these glaciers, adding a thrilling edge to their journey. Furthermore, the view of Baba Kedarnath Dham from Dev Darshani is exceptionally divine and captivating this season, promising a deeply spiritual experience for all visitors. The portals (kapat) of Baba Kedarnath are set to open for devotees on April 22. With the opening of the gates, thousands of pilgrims are expected to arrive at the Dham, marking the formal commencement of this holy pilgrimage. (Source: Local Administration)
0:07

#WATCH | Rudraprayag, Uttarakhand | Due to fresh snowfall in April, the Kedarnath temple complex and its surrounding areas are currently draped in a pristine white blanket of snow. Along the trekking route, glaciers have descended near Chhoti Lincholi and Badi Lincholi. As pilgrims advance from Gaurikund toward Lincholi, they will traverse paths carved directly through these glaciers, adding a thrilling edge to their journey. Furthermore, the view of Baba Kedarnath Dham from Dev Darshani is exceptionally divine and captivating this season, promising a deeply spiritual experience for all visitors. The portals (kapat) of Baba Kedarnath are set to open for devotees on April 22. With the opening of the gates, thousands of pilgrims are expected to arrive at the Dham, marking the formal commencement of this holy pilgrimage. (Source: Local Administration)

ANI

15,161 Aufrufe • vor 2 Monaten

#WATCH | Guwahati, Assam: An FIR has been filed against the Assam Jatiya Parishad (AJP) candidate of the Guwahati Central seat, Kunki Chowdhury, at the Panbazar Police Station for violating the Model Code of Conduct on the polling day, April 9th. She says, "An FIR was filed against me, making some false allegations. So I came here to give a statement... I presented my side and gave my statement. I have full faith in the system, and it will investigate thoroughly..."
0:18

#WATCH | Guwahati, Assam: An FIR has been filed against the Assam Jatiya Parishad (AJP) candidate of the Guwahati Central seat, Kunki Chowdhury, at the Panbazar Police Station for violating the Model Code of Conduct on the polling day, April 9th. She says, "An FIR was filed against me, making some false allegations. So I came here to give a statement... I presented my side and gave my statement. I have full faith in the system, and it will investigate thoroughly..."

ANI

28,172 Aufrufe • vor 3 Monaten

#WATCH | Andhra Pradesh Deputy CM Pawan Kalyan arrives in Delhi to participate in tomorrow’s “Sena Prasthanam… For National Integrity” programme. During the programme, Pawan Kalyan is expected to give direction and guidance to party leaders on the movement’s vision and future course of action. (Source: Janasena party office)
0:13

#WATCH | Andhra Pradesh Deputy CM Pawan Kalyan arrives in Delhi to participate in tomorrow’s “Sena Prasthanam… For National Integrity” programme. During the programme, Pawan Kalyan is expected to give direction and guidance to party leaders on the movement’s vision and future course of action. (Source: Janasena party office)

ANI

34,005 Aufrufe • vor 1 Monat

#WATCH | Haryana: A video went viral where a man was throwing currency notes from his running car in Gurugram. Police file a case in the matter. (Police have verified the viral video)
0:15

#WATCH | Haryana: A video went viral where a man was throwing currency notes from his running car in Gurugram. Police file a case in the matter. (Police have verified the viral video)

ANI

895,111 Aufrufe • vor 3 Jahren

#WATCH | Responding to ANI's question on the future of India-US trade negotiations in view of US sanctions imposed on India, White House economic adviser Kevin Hassett says, "The trade team and the President are disappointed that India continues to fund Russia's Ukraine war. Hopefully, it's a diplomatic issue that will have positive developments soon." (Source: US Network Pool via Reuters)
0:15

#WATCH | Responding to ANI's question on the future of India-US trade negotiations in view of US sanctions imposed on India, White House economic adviser Kevin Hassett says, "The trade team and the President are disappointed that India continues to fund Russia's Ukraine war. Hopefully, it's a diplomatic issue that will have positive developments soon." (Source: US Network Pool via Reuters)

ANI

82,536 Aufrufe • vor 10 Monaten

#WATCH | Defence Minister Rajnath Singh leaves from 7, LKM. Prime Minister Narendra Modi held a meeting with Defence Minister Rajnath Singh, NSA Ajit Doval, CDS and Chiefs of all three Services, at 7, LKM.
0:19

#WATCH | Defence Minister Rajnath Singh leaves from 7, LKM. Prime Minister Narendra Modi held a meeting with Defence Minister Rajnath Singh, NSA Ajit Doval, CDS and Chiefs of all three Services, at 7, LKM.

ANI

197,226 Aufrufe • vor 1 Jahr

#WATCH | Hyderabad, Telangana | After the meeting with SP chief Akhilesh Yadav, BRS working president KT Rama Rao says, "We draw inspiration from you and your party about the manner in which you reached Delhi with 37 MPs. We will bounce back once again and try to come back to power..."
0:18

#WATCH | Hyderabad, Telangana | After the meeting with SP chief Akhilesh Yadav, BRS working president KT Rama Rao says, "We draw inspiration from you and your party about the manner in which you reached Delhi with 37 MPs. We will bounce back once again and try to come back to power..."

ANI

209,095 Aufrufe • vor 7 Monaten

India win the 2023 Women's Hockey Junior Asia Cup, beat South Korea 2-1, in Japan Hockey India announces the players will receive a cash prize of Rs 2 lakhs each and support staff will receive a cash prize of Rs 1 lakh each for clinching their maiden Women's Junior Asia Cup 2023 title (Video source: Hockey India/Twitter)
0:10

India win the 2023 Women's Hockey Junior Asia Cup, beat South Korea 2-1, in Japan Hockey India announces the players will receive a cash prize of Rs 2 lakhs each and support staff will receive a cash prize of Rs 1 lakh each for clinching their maiden Women's Junior Asia Cup 2023 title (Video source: Hockey India/Twitter)

ANI

1,372,498 Aufrufe • vor 3 Jahren

#WATCH | Hyderabad, Telangana: GRP and RPF Secunderabad seize Rs 1,22,70,000 from a person transporting hawala money from Mumbai to Secunderabad (Source: Police)
0:18

#WATCH | Hyderabad, Telangana: GRP and RPF Secunderabad seize Rs 1,22,70,000 from a person transporting hawala money from Mumbai to Secunderabad (Source: Police)

ANI

137,429 Aufrufe • vor 1 Monat

#WATCH | "I understand that India is no longer going to be buying oil from Russia. That's what I heard, I don't know if that's right or not. That is a good step. We will see what happens..." says, US President Donald Trump on a question by ANI, if he had a number in mind for the penalties and if he is going to speak with PM Modi. (Source: US Network Pool via Reuters)
0:16

#WATCH | "I understand that India is no longer going to be buying oil from Russia. That's what I heard, I don't know if that's right or not. That is a good step. We will see what happens..." says, US President Donald Trump on a question by ANI, if he had a number in mind for the penalties and if he is going to speak with PM Modi. (Source: US Network Pool via Reuters)

ANI

841,442 Aufrufe • vor 11 Monaten