pip install spectralquant ✂️ Up to 6.62x KV cache... compression for LLMs and transformers. Same model. Faster outputs. Smaller KV cache. Try now (2 mins): - KV cache integration via Hugging Face's DynamicCache - Three presets: 5.95x (paper), 6.55x (validated), 6.68x (edge) - Mistral 7B / Qwen 2.5 7B / Llama 3.1 8B verified - Pure PyTorch + future CUDA kernel support - Auto-calibration from a bundled corpus 📰 Paper: 💻 Code, quickstart, and benchmarks: #LLM #Inference #PyTorch #OpenSource #MachineLearning #LLM #KVCache #Inferenceshow more

🚀 Self-speculation brings 6.75x real speedup for LLM generation... with SGLang inference! Same model drafts future tokens in Diffusion mode → then verifies them in AR (causal) mode. One model and one KV cache. Just different attention masks. Thanks to perfect alignment, we get 2× longer acceptance lengths than MTP techniques (Eagle-3, MTP, dFlash). We run 2 forward passes… but the 2× higher acceptance means we break even - and with zero overhead from extra drafter, KV cache, or LM head that comes with MTP - those are not free. Last week we released Nemotron-Labs-Diffusion + Tri-mode LLMs! We did continued pre-training on Ministral-3 models by switching attention patterns (block causal bidirectional). Result: one model that runs AR mode, Diffusion mode, and Self-Speculation. Diffusion mode already shows high benchmark accuracy - excited to see what happens when someone beats left-to-right acceptance! 🔥 Github: Paper: SGLang inference: Try the models on HF:show more

#WATCH | Delhi | An AAP worker says, "Dictatorship... will not be tolerated. You are picking up students as if they were terrorists. Yet, those responsible for leaking the papers are not being picked up. The government is failing to apprehend the paper leak culprits."show more

#WATCH | Delhi: Lok Sabha LoP and Congress MP... Rahul Gandhi, along with INDIA bloc MPs, reach Gandhi Smriti as they continue their protest over the NEET paper leak.show more

#WATCH | Muskan from Chandigarh says, "We are here... to support an Army. They are doing so much for us, and we also want to do something for our Army."show more

#WATCH | TVK Chief Vijay meets Tamil Nadu Governor... Rajendra Vishwanath Arlekar at the Lok Bhavan. With the support of Congress, CPI, CPI(M), VCK and IUML, 121 MLAs are now in support of TVK, and its path to form a government in the state is now clear. (Source: Lok Bhavan)show more

#WATCH | Patna | On tweet by Kerala unit... of Congress, ' 'B for Beedi and B for Bihar...' (which stands deleted now), RJD leader Tejashwi Yadav says, "It was a wrong tweet. We don't support it."show more

#WATCH | Delhi: BJP MPs staged a protest outside... the Parliament demanding the resignation of Punjab Education Minister Harjot Singh Bains, over the alleged paper leak in the state.show more

#WATCH | Delhi | BJP leader Smriti Irani says,... "...Vikram talks about India's capacity to be the bridge not only between the global north and the global south, but also to ascend to a new formidable future. My compliments to the author and my gratitude for his service"show more

#WATCH | Kolkata: Former CM and TMC chief Mamata... Banerjee says, “If need be, I myself will go to the CJP protest. They had our support from the beginning, and will continue to have it.” (Source: TMC)show more

#WATCH | Auckland, New Zealand | Prime Minister Narendra... Modi emplanes for India after concluding his three-nation visit to Indonesia, Australia, and New Zealand. (Source: ANI/DD)show more

#WATCH | On Bagram Airbase in Afghanistan, US President... Donald J Trump says, "We're talking now to Afghanistan, and we want it back, and we want it back soon. If they don't do it, you're going to find out what I'm going to do." (Source: US Network Pool via Reuters)show more

#WATCH | Rudraprayag, Uttarakhand | Due to fresh snowfall... in April, the Kedarnath temple complex and its surrounding areas are currently draped in a pristine white blanket of snow. Along the trekking route, glaciers have descended near Chhoti Lincholi and Badi Lincholi. As pilgrims advance from Gaurikund toward Lincholi, they will traverse paths carved directly through these glaciers, adding a thrilling edge to their journey. Furthermore, the view of Baba Kedarnath Dham from Dev Darshani is exceptionally divine and captivating this season, promising a deeply spiritual experience for all visitors. The portals (kapat) of Baba Kedarnath are set to open for devotees on April 22. With the opening of the gates, thousands of pilgrims are expected to arrive at the Dham, marking the formal commencement of this holy pilgrimage. (Source: Local Administration)show more

#WATCH | Guwahati, Assam: An FIR has been filed... against the Assam Jatiya Parishad (AJP) candidate of the Guwahati Central seat, Kunki Chowdhury, at the Panbazar Police Station for violating the Model Code of Conduct on the polling day, April 9th. She says, "An FIR was filed against me, making some false allegations. So I came here to give a statement... I presented my side and gave my statement. I have full faith in the system, and it will investigate thoroughly..."show more

#WATCH | Andhra Pradesh Deputy CM Pawan Kalyan arrives... in Delhi to participate in tomorrow’s “Sena Prasthanam… For National Integrity” programme. During the programme, Pawan Kalyan is expected to give direction and guidance to party leaders on the movement’s vision and future course of action. (Source: Janasena party office)show more

#WATCH | Responding to ANI's question on the future... of India-US trade negotiations in view of US sanctions imposed on India, White House economic adviser Kevin Hassett says, "The trade team and the President are disappointed that India continues to fund Russia's Ukraine war. Hopefully, it's a diplomatic issue that will have positive developments soon." (Source: US Network Pool via Reuters)show more

#WATCH | Haryana: A video went viral where a... man was throwing currency notes from his running car in Gurugram. Police file a case in the matter. (Police have verified the viral video)show more

#WATCH | Hyderabad, Telangana | After the meeting with... SP chief Akhilesh Yadav, BRS working president KT Rama Rao says, "We draw inspiration from you and your party about the manner in which you reached Delhi with 37 MPs. We will bounce back once again and try to come back to power..."show more

#WATCH | Defence Minister Rajnath Singh leaves from 7,... LKM. Prime Minister Narendra Modi held a meeting with Defence Minister Rajnath Singh, NSA Ajit Doval, CDS and Chiefs of all three Services, at 7, LKM.show more