Beyond Code Snippets: Benchmarking LLMs
on Repository-Level Question Answering March 2026
Using this dataset, we systematically evaluate two widely used LLMs (Claude 3.5
Sonnet and GPT-4o) under both direct prompting and agentic configurations. We compare
baseline performance with retrieval-augmented generation methods that leverage file-level
retrieval and graph-based representations of structural dependencies. Our results show that
LLMs achieve moderate accuracy at baseline, with performance improving when structural
signals are incorporated. Nonetheless, overall accuracy remains limited for repository-
scale comprehension. The analysis reveals that high scores often result from verbatim
reproduction of Stack Overflow answers rather than genuine reasoning.
DOI:10.48550/arXiv.2603.26567
Via
Update from "nobody thinks they're the villain in their own story" to "anybody who thinks they're the hero in their own story is probably the villain."
I've eventually, after looking at situations like Spade Cooley, come around to the fact
that it's not bad to support the estate of people who've done horrific things, if the
estate is paying into funds which help the victims. I can "separate the art from the
artist" when the art is helping mitigate some of the damage.
I've also come around (and there's history on Flutterby, eg, of me being
dismissive) to understanding that Michael Jackson was one hell of a singer, and, the
product of a very fucked up childhood, and product of a very fucked up society in how we,
collectively, handled his celebrity.
So I've been kinda looking forward to the upcoming Michael Jackson movie.
But I'm also well aware that... there's some problematic shit here. And somehow I missed
this headline from January of last year, that Michael Jackson Biopic Needs Major
Reshoots After Discovery of Past Legal Agreement with Molestation Accuser: Report.
More recently, Inside the ‘Michael’ Overhaul: $15 Million
Reshoots, Removing Child Abuse Allegations and What’s in Store for Sequels which names
the accuser whose lawyer made sure that there was to be no mention of said accuser in
future films. Decades ago.
(Still)
An(gr)i Bundel @anibundel.bsky.social observed:
I feel like not enough reviewers know they had to remake the entire final
third of the Michael Jackson movie because it falsely exonerated him, and it turned out
the kid’s lawyer foresaw that shit in the 1990s and made sure to include a clause that the
estate could never ever do that on film.
(Still)
An(gr)i Bundel @anibundel.bsky.social
Note I said “remake.” As in, the Jackson estate apparently had *no idea* they
had signed something 25 years ago that prevented them from ever defaming the kid until the
movie was basically finished.
I can't imagine that the estate's legal team somehow dropped this. I would think that the
screenwriters would have been working with these settlement agreements all the way
through.
Were
Abby
@kellylink.bsky.social
This Is Just To Say
I have turned off
the AI features
that were in
the update
and which
you were probably
hoping
to monetize
Fuck you
they were stupid
so unnecessary
and so annoying
UCLA Center for Parking Policy: Minimum
Parking Requirments - A Research Synthesis
Eliminating minimum parking requirements does not eliminate the environmental,
social, and economic harms of parking, but it can reduce their severity. Research indicates
that the early effects of repeal are modest rather than dramatic. After requirements are
lifted, drivers make more efficient use of existing parking infrastructure, and developers
also continue to supply new parking. As a result, the total number of spaces and vehicles
citywide may continue to grow while the number of parking spaces per capita declines over
time. Minimum parking requirements exert a long shadow, having been entrenched in U.S.
cities for more than 75 years. Even after repeal, cities will be dealing with the legacy of
an oversupply of parking for many years to come.
I've only gotten as far as the executive summary, but my takeaway is that we should be
probably aggressively pursuing parking maximums, not merely repealing parking minimums.
Is anybody else having kerning problems reading "Ternus" as "Temus", and thinking about the future of Apple?
(Not a dig at the guy, the MacBook Pro has definitely been rescued from the Ives era. Even if it's hobbled by Liquid Glass )