Did you compare what you’ve built to some of the internals of nuitka? nuitka stops dead code search at the module level, but their exclusion/inclusion flags/rules might steer you towards fewer false positives for common packages.
selcuka 10 hours ago [-]
Cool project. I think false positives are more important than the dead code it misses at this stage. For example, it marks all Django settings as unused variables, probably because settings are typically not imported directly in Django projects. For example:
from django.conf import settings
...
if settings.DEBUG:
...
Currently unref thinks that DEBUG is not used.
Similarly, the `application` variable in asgi.py (or wsgi.py) is passed as an argument to the ASGI/WSGI server executable, but that is also marked as unused.
McAvey 8 hours ago [-]
Thanks, one of my goals was to try and make popular frameworks work well with this tool, and I didn't add Django yet.
I'll make a PR to add some basic Django support which should address both of your points.
pennomi 4 hours ago [-]
I wish they made Django a little less “magic”. Virtually every harness has to create workarounds specifically for Django, and simple things like settings probably don’t need to break standard coding practices.
drcongo 7 hours ago [-]
I got a lot of false positives in a Django project too, but trying it on some non-framework tools I've built worked pretty well. Will keep an eye on this, thanks.
athoscouto 6 hours ago [-]
I've been trying to do something like this internally too. Also started with Python, then moved onto ruff. My goal is a bit broader though. I want to use the dependency graph to detect changed parts of the project to selectively trigger image builds and run tests.
Results are still poor, specially on the tests side. So far the main learning has been that a tidy codebase (dep graph) is essential for it to work properly.
I'm surprised I didn't find more open source tooling tackling this effectively.
McAvey 3 hours ago [-]
Yeah and I think these problems are only increasing in importance in AI times, with code generation being so easy.
tomtom1337 5 hours ago [-]
What is the definition of dead code that Unref uses? I distinguish between apps and libraries. In apps I can see how any code not callable from the entrypoint is dead, and for libraries I guess a definition is any code that is not callable from a public api?
Would be nice to see the definition used placed in the docs/github readme.
McAvey 2 hours ago [-]
Yeah exactly, for apps it's any code that is not reachable from the app entry point, and for libraries is any code not reachable through the public API.
I've updated the README and added a FAQ section in the docs to explain this, thanks for the question!
Thanks, this seems like a very good approach and is indeed very fast! Happy to run this over our codebase and see how it compares against Vulture.
McAvey 9 hours ago [-]
Great let me know how it goes!
The initial version was in Python which was pretty fast as well, decided to try Rust both for speed and to reuse more constructs from ruff/ty for better compatibility with them.
leobuskin 10 hours ago [-]
All of CPython’s stdlib, right? It’s Python dead code checker, not C.
McAvey 8 hours ago [-]
Yes, only the Python code (every .py file in the CPython repo).
The benchmark script is in the repo so you can run it yourself, the results are from my potato computer, will probably be even faster on modern PCs.
I've updated the README to be more clear about that, thanks for pointing that out!
maxkaya_dev 9 hours ago [-]
[flagged]
Shexfer 7 hours ago [-]
[flagged]
olk09oscar 9 hours ago [-]
[dead]
tthanh2k6 7 hours ago [-]
[flagged]
WanderZil 6 hours ago [-]
[dead]
gounscrambler 12 hours ago [-]
[flagged]
musicpublisheri 12 hours ago [-]
[flagged]
Rendered at 19:08:26 GMT+0000 (Coordinated Universal Time) with Vercel.
Similarly, the `application` variable in asgi.py (or wsgi.py) is passed as an argument to the ASGI/WSGI server executable, but that is also marked as unused.
I'll make a PR to add some basic Django support which should address both of your points.
Results are still poor, specially on the tests side. So far the main learning has been that a tidy codebase (dep graph) is essential for it to work properly.
I'm surprised I didn't find more open source tooling tackling this effectively.
Would be nice to see the definition used placed in the docs/github readme.
I've updated the README and added a FAQ section in the docs to explain this, thanks for the question!
Here's the PR - https://github.com/Apakottur/unref/pull/121
Feel free to open an issue on github if you find any false positives/negatives.
Feel free to open a GitHub issue with a few of the false positives: https://github.com/Apakottur/unref/issues
The initial version was in Python which was pretty fast as well, decided to try Rust both for speed and to reuse more constructs from ruff/ty for better compatibility with them.
The benchmark script is in the repo so you can run it yourself, the results are from my potato computer, will probably be even faster on modern PCs.
I've updated the README to be more clear about that, thanks for pointing that out!