Just speaking for myself here, I didn’t realize the power of static typing for years and years because the main languages I used with “static types” were C-style; i.e. type checking without null checking. I was vaguely aware of “better” systems but since I had such a wealth of experience with compile-time type-checking catching ~zero bugs, it didn’t seem worthwhile to investigate, and my toy projects didn’t make it clear what I was missing. Having used Mypy in a large practical situation for a couple of years, one of the MAIN advantages of it is the None checking! I would get probably 60% of the utility out of a type checker that could only check for None.
Having had this experience now it’s much easier to appreciate the practical benefits of systems like Rust and Haskell, where the type checker is doing real work.
That's cool if there's any indication that this could actually be used in production. Namely, what is the compatibility with the ecosystem. The history of Python could be aptly characterized as a series of big problems -> tools promising miraculous solutions -> tools utterly failing to deliver on those solutions because they failed to consider some important part of the ecosystem. We need more than a benchmark graph to suggest that this is a panacea to Python's longstanding performance problems.
Mypy itself uses this extensively; it was developed because typechecking itself was getting too slow. So it’s seen some practical real-world use. It’s not a panacea (heck, it’s barely documented) but it’s an appealing option if you have some clear hotspot in Python that you need to optimize.
There is a very similar tool called Cython that is used by scikitlearn. Coincidentally Cython also solves a lot of python 2/3 problems since it compiles for both.
There’s no error in the original example; in Python, the ability to participate in inheritance diamonds with arbitrary other classes is a feature which must be explicitly documented and maintained. James Knight wrote a really good piece on this many years ago: https://fuhm.net/super-harmful/
The solution is just to avoid inheritance. It’s full of terrible pitfalls in most languages, but Python more than most.
One issue the ecosystem currently has, really (and its not the only one, I believe it's difficult almost everywhere), is that tracking dependency-rot is hard. Unless something breaks outright, you'll never know if a library has been abandoned; and manually checking dozens of github/gitlab repos is expensive and tedious.
Pypi has an api (https://pypi.org/pypi/<pkg-name>/json) that can be leveraged to implement alerts like "this pkg last released 5 years ago, it might be dead!". I guess that's what the "security" package uses already. It would be cool if they added an option to report on this sort of thing.
It's not an "in-joke". It follows conventions set by the Python language:
- "def" instead of "define"
- "cls" instead of "Class"
- "sys" instead of "system"
- "os" instead of "operating_system_facilities"
- "ntpath" instead of "windows_path_names"
- "abc" instead of "abstract_base_classes"
Python aggressively abbreviates almost everything in common use. If it were in the stdlib, perhaps it would be the built-in names "attrs" and "attrib" rather than having dots in them, but the names would indubitably still be very short.
It's an "in-joke" in the sense that it doesn't make sense until you get the joke. There are people in this post discussion asking "What means .s and .ib?" It's not obvious to everyone.
if you have a dict, it maps something to something else. You should be able to add and remove values.
Objects, on the other hand, are supposed to have specific fields of specific types, because their methods have strong expectations of what those fields and types are.
attrs lets you be specific about those expectations; a dictionary does not. It gives you a named entity (the class) in your code, which lets you explain in other places whether you take a parameter of that class or return a value of that class.
Why not? It's aggressively tested and benchmarked to ensure its correct and its performance impact is as close to zero as possible. (It's pretty close.)
Both of those things are very good, even great, but they don't speak to the issue: It's [bad] magic.
In this case, it does at module-load-time what should have been done previously at build-time. (And yes, I know Python normally doesn't have build-time the way e.g. C does.)
I'm working on a large production codebase right now where several of the cowboy-coders here have added all sorts of crazy metaprogramming and it's a PITA.
Let me tell you a story. We had a class decorator that introspected the attributes of the class and added "constants" to the enclosing module providing named string values for each of the class's attributes. It lets us access dict versions of these objects (they're data models) like foo_as_dict[foo_module.SOME_ATTR_NAME_BUT_IN_CAPS] ...
Good luck finding foo_module.SOME_ATTR_NAME_BUT_IN_CAPS in foo_module.py though, because it's not there. (and notice that if the model field name changes ALL of your uses of the "constant" have to be refactored to the new name too, so it's not really as helpful as it might seem in the first place.)
And now every new developer has to ask, "Where are these coming from?" and we get to point out the magic extra-clever decorator. (Try to imagine the horrible wonderful "voodoo" that decorator encompasses... Imagine the innocent newbie encountering it.)
We have a "service builder" thing that takes a bunch of arguments and build a web service object in memory at module-load time. No source for any of that machinery. A bug in the service gives you a traceback running through a bunch of weird generic meta-code.
Did I mention the guy who originally wrote it bailed to a new job a month after he finished it? No one else knows how it works. We can read the code and reverse engineer it, of curse, but that's kind of B.S., no? A junior dev could debug the real service code if it existed, but the meta-code that generates the services is another challenge (that they shouldn't have and the company shouldn't have to pay them to overcome.)
We had unittest that read a JSON file and built a test suite in memory to run a bunch of other tests. They finally let me re-write that to a build-time script that scans the exact same JSON files and emit plain-vanilla unittest modules, that then can be run as normal, and the dev/user can look at the actual code of the test, rather than the meta-code of the test-builder.
The same problem applies to this attr package.
If you're trying to trace into or debug code that uses attr you've got to know attr at least enough to be able to follow what it's doing. It's an in-joke. ;-)
(P.S. I'm a big fan dude. Nice to interact with you. All respect.)
I understand that spooky action-at-a-distance magic can really ruin a codebase's maintainability. But what you've developed here is not a judicious appreciation of its danger and its power, but a blanket aversion to both its risks and its benefits.
An apt metaphor would be, let's say: toast. If you try to make some toast, but accidentally burn your house down, it's understandable that you might want to have your bread un-toasted for a while. But it doesn't make sense to switch your diet to be entirely raw as a result, especially if you eat a lot of chicken.
In python, metaprogramming is like fire. People who haven't seen it before are fascinated before it, and try to touch it. This always goes badly. But that doesn't mean it's useless or we shouldn't have it. There are many things it's good for, if it's carefully and thoughtfully applied.
attrs goes out of its way to avoid any behind-the-scenes effects. It even generates Python code, rather than using setattr(), so that if any error happens in the constructor, you can step through it normally in the debugger, and see it in the traceback. Its semantics are clear and direct. It doesn't have these problems.
I like toast, and a toaster is better than a folded wire clothes hanger perched over the stove burner, but this robot toast-o-matic is too fancy for my kitchen at work. We gotta toast that toast and ain't nobody got time to debug some fancy toaster.
:-) Metaphors. Heh.
Let me reiterate the first part of my statement above, to wit: "this is wonderful and cute as hell".
I like it.
I could use it if it generated complete code for the classes you define. That would actually put to rest my "metaprogramming BAD" problem in this case.
Tucking __init__ source in linecache for the debugger is neat but it's also more magic. You might wind up stepping through code that doesn't exist in any file. Is this wonderful hacky weirdness mentioned in the docs? I didn't see it.
And what about __repr__? No precomputed string template?
Don't imagine that I'm crouching in the dirt tentatively reaching out to the monolith of metaprogramming. I'm into it. I read GvR's "The Killing Joke" for fun. But he called it that for a reason. :-)
I still recall a junior dev slamming into a wall trying to extend a Django HTML form widget. The reason? The cowboys over at Django made it with a metaclass.
We should look at a problem and think, "What's the simplest thing that will work?" Or, more to the point, "Does this problem require metaprogramming to solve?" ("or am I just showing off?" could be the rest of that question...)
HTML form widgets don't require metaclasses. The problem attrs solves doesn't require metaprogramming. It's a simple DSL for data models. Right? Riiiight?
If attrs took in a spec (as Python code or whatever) and emitted all your boilerplate, ready to go, I would love it so much. You would get all of the benefits without any of the downside. I know that's boring and not sexy or fun, but I just want to get this bug fixed and ship it and get on with my life. I'll play with attrs at home, on my own time, but I'm sick of magic at work.
Why would it need to recurse? The method probably just returns a copy of the instance's __dict__. Maybe updating it with the __slots__ and their values.
Can you elaborate? What makes it a "huge dependency to maintain"? Is there anything that the Twisted project can do to make it easier? If this is actually a problem I'd really like to hear from users on the Twisted mailing list and bug tracker.
Twisted is a general purpose library/framework with lots of features. This is the "huge" part. In my previous projects I have used it a lot and appreciated it.
What I was trying to tell is if Scrapy uses only small part of library, it may be possible for developers to use similar constructs from Python's standard library. In any case dependency is dependency and it is always better to minimize code footprint.
Having had this experience now it’s much easier to appreciate the practical benefits of systems like Rust and Haskell, where the type checker is doing real work.