CPython is really quite slow by design. The reference implementation is meant to be obvious and pretty easy to interoperate with C.
I think a RustPython implementation would be pretty cool. You could definitely take that opportunity to worry about performance more than CPython does while also worrying about interoperability more than PyPy does.
Or I'm missing your point and you're suggesting a drop-in replacement for CPython that supports all the same C-based libraries as CPython does.
PyPy is doing really well on compatibility these days. It can use all the C libraries natively, and a lot of the speed issues there have been resolved.
Literally all of them, without any issues? I'm working on implementing support for Ruby's C extensions in an alternative implementation and it's a right slog.
PyPy dev here. Yes to both - all of them and it was a right slog. If it doesn’t work report an issue. We have a proposal looking for funding to make it fast.
Great! I appreciate you sharing this. I checked the PyPy main page, Downloads page, and Compatibility page and while I didn't look exhaustively, nothing mentioned 3.6 or 3.6 features.
From my limited experience with Rust, having support for C-based libraries shouldn't be too hard to do- as Rust (like Go) has support for embedding C. I'm not sure how portable the solution would be- I would guess you would need to compile the VM with the modules you want baked in.
I think not having support for C modules would hamper long term adoption. I would absolutely love to adopt this for my stuff, but off the top of my head- I use uvloop and confluent-Kafka, both of which are largely written in in C. Moving away from those would be hard-ish.
Calling into and out of C (whether dynamically or statically linked) isn't the hard part; the hard part is implementing the somewhat arbitrary interface against which C extensions are written in a way that is functionally compatible with CPython without losing the advantages of not being CPython.
They are probably the fastest dicts and lists we can imagine today. If you really need lists and dicts for your data, you need them and won't gain much. But if you don't need them and instead could use something simpler, an array of C structs would be much faster than a list of dicts.
No, that was my point basically, although I could imagine something like this shipping with some basic libraries and package support, and then having a similar ecosystem to the current python ecosystem.
Python has massive use in a diverse array of fields with lots of educational resources helping beginners. There is and will be for a while lots of Python code. It's increasingly used in business-critical systems, like at Bank of America. That it's built on an unsafe language puts that all at risk of unreliability and security vulnerabilities. Long ago, I wanted to rewrite it in Ada with all safety features on to mitigate that risk while maintaining its convenience. Rust is another safe, systems language with extra measures for temporal errors. So, this project is doing the same thing.
Full security for Python apps would require consideration of each layer of abstraction:
1. User's code in Python.
2. The interpreter and extensions.
3. How these interact.
4. If added for performance or security, any assembly code plus its interactions.
Rewriting Python interpreter in Rust mainly addresses No 2. An example of a method to address all of them would be Abstract, State Machines which can represent simultaneously language semantics, software, and hardware. Tools like Asmeta exist to make them like programming languages. The verification would probably be manual, specialist work. Whereas, Rust's compiler gives you key properties with just annotations for many and working with borrow-checker for a few.
Now but you need an incredible high level of mastery to work on such a big C project so widely deployed.
Rust will allow to safely invite a broader range of contributors, because there are so many things you don't need to check. This also means a smaller number of required tests, and because Rust uses higher level constructs that C, more productivity in general.
So basically, on the long run, more people, able to do more things.
Besides, on of the goals of the main implementation is to stay simple, which is hard to do in C. For those reasons, and because of the potential for unreliability and security, CPython is quite slow.
We can't optimize it, because it would make it too complex.
But with a rust implementation, one can hope to suddenly be able to apply more optimizations.
It's all theorical of course, but it's a nice hope.
There's a mature 30 year old codebase, with huge industry update and tons of major players using it, and some random project to rewrite it in a new fashionable language, that will more likely than not go nowhere (as 99% of such attempts do) and be abandoned when the committers lose interest and recognize how big the full task is.
I'm curious to know that why isn't it the case right now? Isn't that one of the selling points of Rust, being blazing fast? Or perhaps just this particular piece of software has been implemented poorly?
Here the optimism would be around removing the GIL and similar stuff, it is not that rust would make the same architecture magically faster but that it could allow a better one.
Also others have noted that speed was not a primary focus of CPython
It's within 5-10% on average, sometimes faster, usually a little slower, very rarely slower by much. The person you're responding to said "approximately". I feel like this is needless pedantry.
Rust also allows you to make architectural decisions in the name of performance that would be completely unmaintainable in C. See: the Servo project.
Further, it should be possible for Rust to eventually reach a state where it's faster than C because of the lack of pointer aliasing. This opens up a whole host of optimization opportunities, allowing it to get closer in performance to Fortran (which is intrinsically the fastest of the bunch).
What makes you say that? Optimized machine code is for the most part optimized machine code, and unsurprisingly benchmarks tend to come down about even. The bunchmarks game currently has them down as
C vs. Rust: 6 wins for C, one draw, 3 wins for Rust
C++ vs. Rust: 5 wins for C++, two draws, 3 wins for Rust
The wins one way or another are also not by particularly large margins.
This may go on to shift marginally in Rust's favor once a soundness bug related to non-aliasing of references is fixed in LLVM and the compiler can safely leverage some guarantees that Rust provides that C and C++ cannot.
>We’ve had almost three decades to optimize CPython
Assuming that was actively pursued. But if it was, all those other projects (Unladden Swallow, Dropbox's Python project, PyPy, etc, whose intend was exactly to make Python faster, wouldn't have been started).
> [Performance] isn't [...] one of the selling points of Rust
or
> this [...] software has been implemented poorly
It sounds like you're maligning Rust (isn't keeping promises) or RustPython (is implemented poorly), and it's easy to read "implemented poorly" as an attack on the implementers.
Well, it's definitely a possibility. I am not trying to attack them, it might be the case, or it might not. I was just trying to get possible reasons, and that's what came to mind. I'm sure there are other reasons, that's why I asked.
I really don't know how slow it is, I've not done benchmarks, but given that Rust is supposed to be efficient, it certainly can't be that slow, unless the implementation is really poor, I guess. I'm not saying it is because I don't know. If anyone has done benchmarks, please do share!
The reason for why I got the idea that it is slow, is that I believed the parent[1], and people have repeatedly claimed that CPython is slow.
[1] "to eventually get the same or better performance as CPython."
I assume this means that it is slower than CPython, and CPython is already extremely slow according to some people even on this page.
One thing to consider is that CPython isn't slow because of the language it's written in, but because of optimizations it isn't doing (namely JIT, I think). Rust can't do the same things any faster than C can, and an early implementation of Python in Rust isn't likely to be much faster than an early implementation in C. Rust has the potential to make certain classes of optimization easier, eventually.
For an example of the huge developer productivity multiplier Rust is for this kind of thing check out what Yorick Peterse has done in his spare time with Inko: https://gitlab.com/inko-lang/inko
Even if RustPython is faster at the architectural level (which it probably does, CPython is infamously slow), CPython has the advantage of a mature codebase and two and a half decades of accumulated performance tuning.
>and two and a half decades of accumulated performance tuning.
Or two and a half decades of performance neglect.
I've read about things like dicts, sort and the like getting faster implementations, but I've never seen a big effort to make CPython faster in general. In fact the first versions of 3.x were even allowed to regress to slower than 2.x.
This is certainly the case when it comes to startup time (an important thing in a scripting language). On my linux system, python2 starts in about 10ms, python3 takes 20-25 ms.
Python with Rust as its foundation sounds like the best idea ever.
I’m curious to know whether or not it would be possible (or reasonable) to eventually get the same or better performance as CPython.