Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Introducing ExtractBench, the most comprehensive benchmark for information extraction from complex enterprise documents. The latest models are pushing the frontier of coding and knowledge work, but surprisingly they still struggle on complex doc extraction tasks in production. A well-tuned extractor must parse multi-page filings without dropping rows, emit exact...

71,372 Aufrufe • vor 6 Tagen •via X (Twitter)

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

We’re open sourcing the first document OCR benchmark for the agentic era, ParseBench. Document parsing is the foundation of every AI agent that works with real-world files. ParseBench is a benchmark that measures parsing quality specifically for agent knowledge work: ✅ It optimizes for semantic correctness (instead of exact similarity) ✅ It has the most comprehensive distribution of real-world enterprise documents It contains ~2,000 human-verified enterprise document pages with 167,000+ test rules across five dimensions that matter most: tables, charts, content faithfulness, semantic formatting, and visual grounding. We benchmarked 14 known document parsers on ParseBench, from frontier/OSS VLMs to specialized parsers to LlamaParse. Here are some of our findings: 💡 Increasing compute budget yields diminishing returns - Gemini/gpt-5-mini/haiku gain 3-5 points from minimal to high thinking, at 4x the cost. 💡 Charts are the most polarizing dimension for evaluation. Most specialized parsers score below 6%, while some VLM-based parsers do a bit better. 💡 VLMs are great at visual understanding but terrible at layout extraction. GPT-5-mini/haiku score below 10% on our visual grounding task, all specialized parsers do much better. 💡 No method crushes all 5 dimensions at once, but LlamaParse achieves the highest overall score at 84.9%, and is the leader in 4 out of the 5 dimensions. This is by far the deepest technical work that we’ve published as a company. I would encourage you to start with our blog and explore our links to Hugging Face to GitHub. All the details are in our full 35-page (!!) ArXiv whitepaper. 🌐: Blog: 📄 Paper: 💻 Code: 📊 Dataset: 🎥 YouTube:

Jerry Liu

108,093 Aufrufe • vor 4 Monaten

Today, Box is announcing major new AI agent capabilities to let customers tap into the full value of their unstructured data. First, we’re announcing all new updates to the Box AI Studio to make it even easier to build AI agents that tap into your enterprise content for any job function, business process, or industry specific use case. We are also expanding our set of foundational agents that customers will be able to use to work with their enterprise content, including new features like search and research on unstructured data. Next, we’re announcing Box Extract to enable customers to use AI agents seamlessly for complex data extraction from any type of document or content. This makes it easier than ever to pull out data from contracts, invoices, research data, marketing assets, medical charts, and more. Finally, we’re introducing Box Automate, a new workflow automation solution within Box that lets you deploy AI agents across enterprise content-centric workflows. With Box Automate, you can design your business process in a simple drag and drop builder and then drop in AI agents at any step in the process. This ensures agents execute tasks at the right steps in a workflow every time. Best of all, our AI agents and workflow tools are designed to work across any system our customers work within, whether it’s leveraging pre-built integrations, Box APIs, or the new Box MCP Server. Ultimately, all of these capabilities come together to transform how companies can work with their enterprise content. Software has historically only been good at automating work that deals with structured data, which is why ERP, CRM, and HR systems have been mainstays of enterprise software for so long. The data in these systems fits neatly into a database, and the workflows are very ripe for automation. But it turns out most of the work in the world deals with unstructured data. It’s ideating through research documents, working with a client on contracts, reviewing details for a new product launch, looking at a patient’s healthcare record to make a diagnosis, working through due diligence documents for an M&A deal, and so on. For the first time ever, we can begin to bring all new insights and automation to this work with AI agents. At Box, we’re incredibly excited to be on this journey to help customers transform how they work with their most important data.

Aaron Levie

91,863 Aufrufe • vor 11 Monaten

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

AK

249,798 Aufrufe • vor 3 Jahren

Zack Polanski speaking at the Bakers, Food and Allied Workers Union, "The government are very good at recognising the problems, at recognising the crisis, the supply chain issues, the energy crisis in Iran, the energy crisis from Ukraine" "But very rarely do they seem to have solutions, things to actually do about it. And when they do have solutions, rarely are they solutions of the scale we need" "So if we look at the energy crisis, for instance, we've heard recently that energy bills in this country could go up 200 pound per year, on average for a household" "That's completely unacceptable" "And far too often I don't hear the solutions from the government that are just so obvious to ramp up our investment in renewable energy to make sure that we're insulating every single home in Britain that needs it" "So it is both warm in the winter and cool in the summer as well as creating hundreds of thousands of good green jobs that could be in public sectors that could be unionised so people are paid properly and treated with dignity and, and care and to remove the subsidies from the fossil fuel companies" "The same people who are destroying our planet should not be getting subsidised by the government at a time when we're in the climate crisis" "But as you know, well it's not just an energy crisis, it's a crisis for food too. Because what we've seen in Iran or implicated by Iran is a fertiliser crisis" "We know how devastating and damaging that already is for our supply chains and for the food that we produce" "And this badly needs intervention, it badly needs help. And what did we see this government do? Well, they cut tariffs on chocolates and biscuits" "Now don't get me wrong, there is room to do this and that will provide a small relief for some families" "That's not a long term plan for UK businesses and UK food production" "That's not a Long term plan to invest in resilience and in our food supply chains, in our energy" "It's not a long term plan that takes these issues seriously, not in the next few weeks or months, but goes we need to fundamentally rethink our systems change and how we provide food security as one of the most fundamental things in our society"

Farrukh

31,310 Aufrufe • vor 2 Monaten

We’re launching Optima. Now anyone can create a custom benchmark for their use case, leveraging Artificial Analysis’ leading research and platform Building and running benchmarks is difficult. We have distilled Artificial Analysis’ research and experience developing benchmarks into Optima, a new platform for benchmarking models on your own workloads and comparing performance, speed and cost efficiency. Optima allows you to find the best model for your task, or an equally performant alternative to your current setup at 10x lower cost or time per task. We’ve integrated Artificial Analysis' research and experience in benchmarks across the Optima workflow: ➤ Build benchmarks based on your own data and use cases: There are three ways to build a benchmark with Optima. Upload an existing evaluation dataset from your own files or Hugging Face, or import agent traces from platforms including Arize AI, Braintrust and langfuse.com. Install the Optima skill to build a benchmark using context from your coding environment and previous sessions. Or simply describe your use case and provide example inputs and outputs, and Optima will build the benchmark for you ➤ Run across the latest models: Run the same benchmark across leading models in a single click, and keep your leaderboard up to date as soon as new models are released ➤ Bring Artificial Analysis grading to your own benchmark: Evaluate responses against objective rubric criteria or using the same pairwise judging approach used for Artificial Analysis benchmarks including GDPval-AA and AA-Briefcase. For pairwise judging, select your preferred responses from a sample and Optima uses those preferences to rank models across your test set ➤ Compare performance, cost and time efficiency: Optima measures more than model performance. Cost per Task and Time per Task are tracked alongside benchmark scores, with category-level results and support for custom metrics, allowing you to compare the tradeoffs between models for your specific use case Ahead of launch, here are examples questions our beta testers answered with Optima: ➤ Which model can save me 10x the cost without a meaningful decrease in quality for my finance & accounting agent? ➤ Which model best matches the writing style of lawyers for my legal agent? ➤ Which model can best identify different elements in my custom image dataset? Optima is available today. Build your own benchmark at

Artificial Analysis

122,197 Aufrufe • vor 4 Tagen

60 years ago, Ronald Reagan gave Americans a warning that has stood the test of time. This is from Reagan's "Time for Choosing" speech: "If we lose freedom here, there is no place to escape to. This is the last stand on Earth. And this idea that government is beholden to the people, that it has no other source of power except to sovereign people, is still the newest and most unique idea in all the long history of man's relation to man. This is the issue of this election. Whether we believe in our capacity for self-government or whether we abandon the American revolution and confess that a little intellectual elite in a far-distant capital can plan our lives for us better than we can plan them ourselves." "You and I are told increasingly that we have to choose between a left or right, but I would like to suggest that there is no such thing as a left or right. There is only an up or down--up to a man's age-old dream, the ultimate in individual freedom consistent with law and order--or down to the ant heap totalitarianism, and regardless of their sincerity, their humanitarian motives, those who would trade our freedom for security have embarked on this downward course." ... "Well, I for one resent it when a representative of the people refers to you and me--the free man and woman of this country--as 'the masses.' This is a term we haven't applied to ourselves in America. But beyond that, 'the full power of centralized government'--this was the very thing the Founding Fathers sought to minimize. They knew that governments don't control things. A government can't control the economy without controlling people. And they know when a government sets out to do that, it must use force and coercion to achieve its purpose. They also knew, those Founding Fathers, that outside of its legitimate functions, government does nothing as well or as economically as the private sector of the economy." ... "They say we are always 'against' things, never 'for' anything. Well, the trouble with our liberal friends is not that they are ignorant, but that they know so much that isn't so." ... "Mr. Democrat himself, Al Smith, the great American, came before the American people and charged that the leadership of his party was taking the part of Jefferson, Jackson, and Cleveland down the road under the banners of Marx, Lenin, and Stalin." ... "Our natural, inalienable rights are now considered to be a dispensation of government, and freedom has never been so fragile, so close to slipping from our grasp as it is at this moment." ... "You and I know and do not believe that life is so dear and peace so sweet as to be purchased at the price of chains and slavery. If nothing in life is worth dying for, when did this begin--just in the face of this enemy? Or should Moses have told the children of Israel to live in slavery under the pharaohs? Should Christ have refused the cross? Should the patriots at Concord Bridge have thrown down their guns and refused to fire the shot heard 'round the world? The martyrs of history were not fools, and our honored dead who gave their lives to stop the advance of the Nazis didn't die in vain. Where, then, is the road to peace? Well, it's a simple answer after all." "You and I have the courage to say to our enemies, 'There is a price we will not pay.' There is a point beyond which they must not advance." ... "You and I have a rendezvous with destiny. We will preserve for our children this, the last best hope of man on Earth, or we will sentence them to take the last step into a thousand years of darkness." Doesn't it feel like Reagan is speaking to us TODAY?

Kyle Becker

104,422 Aufrufe • vor 2 Jahren

Why did Ian Whiffin agree to give expert testimony on Jen McCabe’s cellphone extraction, when the state refused to let him look at or even give him the full, original extraction OR its verification hash? A thread🧵 Full cellphone extractions, sometimes called forensic images, generate what’s called a “hash value”, which serves as a unique digital fingerprint necessary for ensuring the integrity of data. Any discrepancy between the hash values indicates tampering with or corruption of evidence, alerting forensic examiners to potential issues with the evidence. Hash verification is a fundamental principle and a rather ubiquitous practice in the world of digital forensics, where data validation and verification are key. It is the gold standard across the industry, and has also become so in the courtroom, whereby admissibility of digital evidence is determined by its relevance, authenticity and reliability. In court, the hash value can be used to demonstrate that the evidence has not been altered since its collection, and is a universal way for experts to authenticate and validate the reliability of data for the trial Court. But, an extraction that’s missing a hash value altogether is a huge red flag. 🚩 Who removed the hash value? And why? It’s necessary to the chain of custody, and as Gaurino and Tully would be well aware, it’s also an element of the data that an expert would require in order to verify and validate it. There’s no “good” reason for why someone would remove a hash value, and the extraction can’t be characterized as a forensic image as its origin is unknown. This was a deliberate step taken to hide something, which one could argue shows consciousness of guilt. If the data are true and accurate, why would you bring their integrity into question by removing the hash? However, if the data were altered or tampered with, and let’s say, hypothetically speaking, you wanted a digital forensic expert to provide testimony supportive of your “Google search” theory. . . In that hypothetical, you’d have to remove the hash value. Otherwise, the expert would immediately detect that the data were altered, as they would not be able to verify the hash against the original. #KarenReadTrial #JusticeForJohnOKeefe #FreeKarenRead #CantonCoverup #PoliceCorruption #KarenRead #Cellebrite #DFIR

Olivia

270,376 Aufrufe • vor 1 Jahr

Scale alone is not enough for AI data. Quality and complexity are equally critical. Excited to support all of these for LLM developers with Snorkel AI Data-as-a-Service, and to share our new leaderboard! — Our decade-plus of research and work in AI data has a simple point: scale alone is not enough. AI success is all about the quality, complexity, and distribution of data—in addition to volume. We’re excited to be powering leading LLM developers with Snorkel AI Expert Data-as-a-Service, our white glove service for custom, expert-level AI datasets—and to now preview some of what we’re building via our new Expert Data Leaderboard (🔗 in 🧵) + upcoming OSS dataset releases! Snorkel Expert Data-as-a-Service is built to meet the rapidly evolving data needs of the agentic AI world—where success is built on the quality, complexity, and distribution of datasets, in addition to size and scale. This kind of high-quality, frontier AI data can only come from a union of technology and human expertise. With Snorkel Expert Data-as-a-Service, we’re powering frontier LLM developers across agentic, expert knowledge, reasoning, coding, multi-modal, and other task types via the combination of these two key components: - (1) The Snorkel Expert Network: A global team of subject matter experts focused wholly on specialized knowledge–spanning thousands of topics in STEM/academic, vertical/professional, and consumer/lifestyle domains. - (2) Snorkel AI Data Development Platform: Our unique programmatic data curation and quality control platform, accelerating and improving expert authoring and review through principled techniques developed over the last decade of R&D. Now: we’re incredibly excited to showcase some of the power of Snorkel Expert Data-as-a-Service via the new Snorkel Leaderboard—putting frontier models to the test in complex, agentic, and reasoning settings inspired by real industry scenarios (not esoteric puzzles)! We’ll be releasing new leaderboards and accompanying expert-verified open source datasets (coming soon!) regularly. To start, we’re sharing three initial ones in preview: - SnorkelFinance: Q&A over financial documents requiring agentic tool-calling and reasoning - SnorkelUnderwrite: Agentic insurance tasks requiring industry-specific reasoning and tool use - SnorkelSequences: Mathematical tasks requiring compositional multi-step reasoning

Alex Ratner

495,851 Aufrufe • vor 1 Jahr

Sayed won his primary and people are already blackpilling. The thing we need to remember about this sudden "success" of socialism is that it is entirely driven by demographics. You'll notice that all of the new shiny faces of socialist America are all Third World invaders. AOC, Ilhan Omar, Milat Kiros, Francesca Hong, Abdul El-Sayed, Darializa Avila Chevalier, Rashida Tlaib, the list goes on. These are not Americans, they are invaders with the ideology of replacement, extraction, and consumption. This has never been an organic rise in the popularity of Socialism, this is a rise in demographic replacement and the promise of expropriating what Americans create and giving it to the teeming biomass of the global favela. This is all just downstream of the Civil Rights Act and the Hart-Celler Act letting in intruders and giving them entitlements over Americans. But my friends, never blackpill. Think about it, they entirely rely on us to produce and create so they can feed on us like leeches. They want to expropriate what we have because they are incapable of productive achievement. Even if they win and banish all "evil colonizer whites" to some gulag in remote Alaska, within 20 years it will be a productive and stable society. They will demand access to the prison they sent us to, because they will have turned what we had into a stinking mud hole, because that is all they are capable of doing. We banished our dysgenic European prisoners to one of the most hostile islands in the world (Australia), and it's now a first world nation that people get on boats made of tires to try and reach. They would have to execute all of us, and then they would still lose because they would have nothing left to feed on. Something tells me if they start trying to liquidate the white Kulaks, we won't be going quietly into the night. We are still the most disagreeable and heavily armed white population in the entire world. They can only push Socialism because we insist on continuing to let them import ethnic enclaves and garner votes by promising to give them our stuff. This is all a choice and at any moment we could simply choose to stop it. It is a choice to let invaders hold political authority in our country that our ancestors built for us, let's make a different choice.

The American Tribune

23,634 Aufrufe • vor 12 Tagen

Our democracy is dying and we are rapidly descending into one party rule. We are left without a credible opposition and a Teal cross-bench with its mouth firmly shut on the slow removal of our right to question the government. The changes that Labor are planning to make in its Freedom of Information (FOI) amendments will further destroy what was already an imperfect system. Secrecy is the key to corruption. Intrinsic to the slow and deliberate removal of our rights, privileges and freedoms. So much happens under the cover of cabinet secrecy, of the quiet respect both sides of politics show each other in not exposing their worst deeds. Like the political class is more afraid of exposure than doing a poor job. It feels that way in Australia because it absolutely IS that way. Gone are the statesmen of old. Those that acted in the national interest. The current crop – Albanese and Ley – don’t deserve our trust. There has been more than enough evidence of corruption and cover-up from both sides, as they just have their turn, or wait for their turn, and nothing really changes. It’s baked in. By design. Ley did a little puff piece on the proposed imposition of a cost to access an FOI, which is a problem, but far from the biggest issue. The two major crimes being committed on the people by the passage of this bill will be the loss of the ability to request information anonymously, and provisions to deeply protect ministers from scrutiny through FOI. A bad convention formalised in law. Secrecy of any kind is abhorrent in a democracy. Without proper information, the public can’t make good electoral decisions. Without the ability to expose information by way of right, it is impossible to hold the government to account. As there is no opposition and scantly any capable media scrutiny. Those responsibilities now fall to members of the public, and in particular, whistleblowers from within the system. Whistleblowers, or anyone who does not toe the line in this country are absolutely pilloried. That rings true in all aspects of our government, and within the political parties that hold the system up. Losing anonymity in requesting documents from government, when paired with the pending loss of anonymity on social media is a one-two-punch that knocks out the last frontier of scrutiny. Our last hope to right the system through the system. Absolutely terrifying. Moreover, the biggest and most important decisions are made in cabinet, and by ministers. Without the ability to ask the big questions of the big players we are left up the proverbial creek, without a paddle. I am sure Sussan will be requesting some minor amendments to cost, but happy to waive the rest through, because she wants to have her turn at it later. We are poorly served by our parliament, we are right to question them, their integrity and intent. When we hear about our involvement in the bombing of Iran from the Americans, and negotiations with India to build 1 million homes here, from India… questions over the ISIS brides and everything else we learn by accident… we must shout our displeasure at this secrecy from the rooftops – while we still can. There must be a public outcry over this breach of faith. Our nation is slipping away from us, like sand through the hourglass. One grain of truth at a time. I just want Australia back.

Matthew Camenzuli

13,891 Aufrufe • vor 11 Monaten

We’re excited to finally introduce Kled Special Tasks, the final major feature included in the V2 app update. Users will now have access to a fully interactive terminal where they can view and complete domain specific upload tasks directly from enterprise buyers. These tasks can be region locked and person specific. For example, PhD students at Stanford might be prompted to upload their coursework or research materials and get paid for it. Our first domain specific task will focus on homework collection from high school and college students across Europe and the United States. Students will verify their emails and academic credentials directly within the app. We’ve built labeling workflows to ensure all uploaded content meets our criteria, and participants will receive weighted payouts based on the value of their submissions. We’ve already built a network of over 3,800 students from Stanford, MIT, UIUC, Rutgers, and Duke who will be actively onboarded to contribute content. Kled will work hand in hand with our research division, HADES, to justify the large scale purchase of this homework content. Several enterprise buyers have already expressed interest, each confirming that academic data from students represents a growing multi year industry requiring a continuous flow of fresh material. Kled Special Tasks also gives us the ability to internally identify valuable content types, issue calls for specific datasets, and collect 1,000-2,000 unique samples per task. We can then package these datasets into specialized data packs that our sales team will use to pitch directly to AI labs and enterprise clients with matching data needs. This will be one of our most powerful tools for expanding Kled’s buyer network. All of this will be fully available in the V2 update. We’re excited to show just how advanced our segmentation and data validation software has become as we bring this release to market.

Kled AI

74,618 Aufrufe • vor 9 Monaten

Hi GoBid as a proudly South African company we at View4You disagree with the fact that people cannot run diagnostics, check the car to see if it starts check for any warning lights, check fluids on your cars. We understand moving them is not doable like all other auctions. Our founder recently reached out to your ambassador mumbo repairs and asked Him why your cars are not auctioned with a report stating the truth about the car as we have come across cases of possible mileage fraud and a case of a cloned car at GoBid. The response we received was that it would be difficult as the quantities are large and it would be costly. He also stated these cars are from various banks insurers etc. He further stated that GoBid will allow us to do our own inspections before we bid. His answers are acceptable and understandable. However, they do depict or show a lack of or no interest in protecting the consumer. Now today we were stopped from running a diagnostic, we were not allowed to get the car started and we were also not allowed to check the fluids. How can people bid confidently with these limitations? Now our humble request is, please permit us to run our checks thoroughly to ensure we know what we are bidding on. All other auctions permit such and it’s only you who does not allow it. A diagnostic test is crucial in the process of practicing due diligence, if you do not want us to run a diagnostic, can you please run it and place the report on the windscreen of your cars that can be diagnosed? You can contact us at [email protected] and we can run it on all your cars if you do not have the capacity to do so. Thank You.

VIEW4YOU

30,414 Aufrufe • vor 3 Monaten