Actually its average because it's 5th on the leaderboard. GPT 5.5 and Opus 4.8 outperformed it. At 5-8x the cost, you would expect better! https://www.endorlabs.com/research/ai-code-security-benchmar...

As TFA says

> Two findings may help explain these average results. > Timeouts > Highest observed cheating

That's why it's 5th on the leaderboard - they give it a fail for every timeout and for every time it gives the correct answer because it knows it.

That's insane

While GitHub needs to invest in finer grained permissioning, I do think there’s lots of lessons for companies building with and customers using GitHub App based deployments. Jotted down my thoughts here https://www.endorlabs.com/learn/when-coderabbit-became-pwned...

A few days ago we published a poll asking how much time it typically takes a developer to investigate an OSS vulnerability reported by an SCA tool.

About 70 people responded, a good mix of security and engineering. Here are some interesting insights.

24% reported it takes less than 2 hours 55% reported it takes more than a day

Most of the 24% were security, and most of the 55% were software engineers.

This started a somewhat...heated discussion on our internal Slack.

What do you think is the reason for the difference in perspective on this?

Leave your thoughts in the comments.

This video is hilarious!