The first open-source implementation of the paper that will change automatic test generation is now available! In February, Meta published a paper introducing a tool to automatically increase test coverage, guaranteeing improvements over an existing code base. This is a big deal, but Meta didn't release the code. Fortunately,... we now have Cover-Agent, an open-source tool you can install that implements Meta's paper to generate unit tests automatically: I recorded a quick video showing Cover-Agent in action. There are two things I want to mention: 1. Automatically generating unit tests is not new, but doing it right is difficult. If you ask ChatGPT to do it, you'll get duplicate, non-working, and meaningless tests that don't improve your code. Meta's solution only generates unique tests that run and increase code coverage. 2. People who write tests before writing the code (TDD) will find this less helpful. That's okay. Not everyone does TDD, but we all need to improve test coverage. There are many good and bad applications of AI, but this is one I'm looking forward to make part of my life.show more

Hey 👋, one of the Cover-Agent creators here. I've recorded 5 minutes video explaining and reviewing TestGen-LLM and Cover-Agent:

Great job and thank you!

It's 630 why are you giving me more work already Santiago I haven't even had my coffee yet

lol

As someone very much involved in testing, this is quite interesting to me. I want to look more into it and explore the quality of the tests. Though, I’m a bit skeptical that this will be replacing someone having to write these tests, the hope is that it at least provides a good starting point for someone.

I'm curious - how does the tool know what the correct behaviour is? Or does it assume that the current implementation is already correct? (Which, to be fair, is how things like Jest snapshots work anyway)

As someone who doesn’t like TDD..

@ponnappa fyi

This can be especially helpful for ML projects. I usually write tests when I bring my product to production, not when starting the project. So this would be a huge time-saver, thanks for sharing.

So, if I understand correctly, the primary purpose of these tests is to safeguard against unintentional changes, right?