> every line of code I write myself is a commitment
That's true. However:
Every dependency you add to your project is also a commitment.
When you add a dependency, you're committing to deal with the fallout if the library you're pulling in gets stale, or gets taken over by an incompetent dev, or conflicts with something else you're using, or just plain disappears. If you add a dependency for just a few lines of code, you're making a way bigger commitment than if you'd just copy/pasted the code and maintained it yourself. That's why so many people are shaking our heads at a 17-line dependency. It's way more risk than it's worth. If you need a better stdlib for your language (some of us write PHP and feel your pain) then find one library that fills in the gaps and use that.
> If you add a dependency for just a few lines of code, you're making a way bigger commitment than if you'd just copy/pasted the code and maintained it yourself.
This is a problem with NPM, not with dependencies. With different package management systems with stable builds and lockfiles, then you pin to a specific version and there is no way upstream can cause problems. A lockfile is a pure win over vendoring.
Yes, there is. Just like NPM's left-pad case.
The owner of the package remove the package from the repository. It doesn't matter if you pin to any version if there is no longer a code to download.
The only way to prevent this is to have your own local server for third party package repository.
You're talking about the purely technical aspects of package management that keep your build from breaking. My point is that there's a lot more to it than that. Lockfiles do not keep software from requiring maintenance. Introducing a dependency means handing off a chunk of your software project to someone else to maintain. If they do it wrong, you're on the hook for it.
For example, I was a maintainer for an admin UI component of the last version of Drupal core, and we decided to pull in a query string handling library written by Ben Alman. It was a good decision, and it hasn't caused any problems. But it still meant we were trusting him to maintain part of our codebase. It was also an implicit commitment to every user of Drupal that if Ben quit maintaining that library, we would step in and fix any problems that came up. You don't get rid of that commitment with a lockfile.
Here is an oversimplified model to illustrate my basic point:
A dependency introduces some constant amount of risk (d) that does not vary with the size of the dependency. Every line of code you write yourself also introduces a much smaller constant amount of risk (y).
If you introduce a separate dependency for every line of code in a 1000-line project, your risk is 1000d.
If you can pull in someone else's code for the whole thing and don't need to write any code yourself, your risk is d.
If 200 lines of your code can be replaced with an external library, your risk is d + 800y.
I think the real disagreement here is over the value of d. My experience leads me to put the value of d pretty high relative to y, so to me 1000d is the worst possible case. If someone sees d as equal to y, then they'd see dependencies as no problem whatsoever.
(Obviously in reality the risk of a dependency is not really constant - it's probably more like d + 0.1y or d + 0.01y or whatever, since a 10-line dependency is less risky than a 1000-line dependency. Hopefully my point still stands.)
You can't escape problems by bundling specific library versions. You just get a different set of problems. When you require a specific version of a library, you're making your code incompatible with anything that requires a higher or lower version of that library. You're also assuming there will never be a security fix that requires you to update your dependency.
...you're making your code incompatible with anything that requires a higher or lower version of that library.
Actually that's not correct when using node/npm (or anything else in the ecosystem like browserify). That is one of the impressive things about this platform: any number of different versions of the same module can be required by the same app. It would be nuts to do that in your own code, but as long as the craziness is in your dependencies it really doesn't matter.
And that kind of works in a dynamic language. You could make it work in a statically-typed language, but then the problems will become more apparent. If X depends on Y and Z1 and Y depends on Z2, and Y exposes an object created by Z2 in its public API, the X developers might look at the Y api docs and try to call Z1 functions on that Z2 object! Worst of all, it might very well work in the normal cases, and the issue might not be noticed until it's in production.
Using multiple versions of the same library is code smell. It's a stability issue, a security issue, a complexity-breeder, and an absolute nightmare for packagers.
Yeah I'm sure it sucks for distro packagers. Why are they using npm, though? It's not designed for their use case.
Actually though you're just talking about some bugs in X, or possibly some design flaws in Y. Passing [EDIT, because bare objects are fine:] class instances around like that is the real code smell. So much coupling, so little cohesion. We call them "modules" because we like modularity.
It's not that packagers are using npm. It's that they might want to package a node application for their distro's package system, and now they have to sift through thousands of npm packages (already a nightmare). They can't just make a system package for every npm package, not just because that would violate packaging guidelines for any reasonable distro, but because one project can pull in multiple versions of a single package. The Nix and Guix crews can handle that (and that they can is as much of a bug as it is a feature).
There is no clean way of packaging a typical node application.
Passing class instances around like that is the real code smell.
Often, yes, but not always. Allowing fine-grained control over performance is one good reason that a library might expose class instances from one of its dependencies.
Is Node.js itself appropriate for packaging? I think maybe not. It changes really quickly, and has done for some time. Anyone coding in Node installs the particular versions she needs without regard to the distro. Most Node modules are just libraries installed in and for particular projects. There are tools written in node, but for the most part they focus on coding-related tasks that also tie them to particular projects, e.g. beefy or gulp. There's no need to install such tools above the project level, and certainly no reason to install them on the system level.
A distro that still packages python 2 (i.e. all of them) has a particular "velocity", and therefore it has no business packaging Node or anything written in it. Maybe a distro could package TJ's "n" tool (helpfully, that's written in bash rather than node), which would actually be handy for distro users who also use Node, but that's it.
I'm not talking about packaging node libraries for developers. No node developers are going to use system packages to install their libraries. What I mean is packaging applications written in node for end users.
For example, you can install Wordpress on Arch with `pacman -S wordpress' and you'll have a managed wordpress installation in /usr/share/webapps/wordpress. Then you just edit some wordpress config files, set up your http server to serve php from that directory, and you have a wordpress blog.
It would be nice to be able to do the same with Ghost.
Ghost may be a special case. I wasn't familiar with it, but I just attempted to install in an empty directory without success. The first time I ran "npm i ghost", with node v5.9, it went into an infinite loop creating deeper and deeper ".staging/ghost-abc123/node_modules/" sub-directories of node_modules, which seems an... odd thing to do. After killing that, I noticed that they recommend Node LTS. Fair enough. I ran "sudo n lts", then "npm i ghost" again. This time, I didn't have to kill it because the preinstall script errored out. Based on the log, this script is installing semver, then requiring a file that can't possibly exist at preinstall time. Both of those are obnoxious, but at least it's possible to install semver.
I'm sure if I look hard enough there are some silly idiosyncratic steps one might take to install this module. Suffice it to say that it's not installing the "npm way", so it's misguided to blame npm for packaging difficulties.
More generally, I can certainly understand distro packagers' refusal to recreate towering pyramids of node dependencies in their own package system. Some lines have to be drawn somewhere, and "major" modules must bundle many of their dependencies when packaged for distros. If module maintainers don't do this, and distro packagers can't, then the modules can't be packaged.
Or you could have all the bugs introduced in everyone's hand-rolled implementations of it. I'll take multiple versions of the library instead. It's much easier to track issues and submit patches to update their dependencies later.
> Or you could have all the bugs introduced in everyone's hand-rolled implementations of it.
Only one buggy implementation per project. Compare this to including the same
library in dozen different versions, because dependencies have their own
dependencies. And you can neither track the versions nor update them.
More importantly, if you only use a specific version of a library, you're opting out of literally every one of the advantages of micro-libraries that people claim they offer. Tying yourself to a single version is the same as copy-pasting the code right into your project, except it doesn't force you to look at the code and vet it.
And writing the code yourself instead of taking on a dependency solves none of these problems. Your code becomes incompatible with anything, because you wrote it yourself. And you are responsible for making any security fixes yourself.
You just get a different set of problems. When you require a specific version of a library, you're making your code incompatible with anything that requires a higher or lower version of that library. You're also assuming there will never be a security fix that requires you to update your dependency.
If there is a security fix, you should bump your dependency by hand, the other problems that you pointed out do not exist in Node (and it's about time they disappear in Java)
I'll admit I'm at least a little tarnished in my practices due to time spent in enterprises where external dependencies require 6 manager sign offs and a security team exemption, but if this were the case that you didnt want updates to the package, just that one version that worked -
If its just a few lines of code, just copy the thing into your code base? throw a comment in saying "came from xxxx" so anyone reading your code knows that it might look like a overly generic function because it is.
...and then the publisher pulls their library off npm, and another shows up and drops one of the same name in its place, with compatible version numbers (by happenstance or otherwise).
That's exactly the problem the parent comment suggests we focus on fixing. Once a library is published, npm shouldn't allow anyone to use that name even if the library is pulled.
True, but it's common to have requirements of the form "^1.0.0" (especially since this is the default of npm i --save). It's easy to publish a new version that would be installed by a project declaring a dependency in this form.
Maintaining a dependency on a library should be much less effort than maintaining 17 lines of code. If it isn't that's a deficiency in your dependency infrastructure.
If you have 100 dependencies, then that's a 100 projects you need to follow and understand the updates for. The minute your dependencies bring in their own dependencies then you start having troubles keep up or even keeping track of the updates. The 17 lines of code you pull in is in most cases a one time deal, having it in a third party library means that you need to keep track of that library for ever.
Honestly, and this is maybe just me being biased against JavaScript, then this is what happens when you pick a language fully knowing it's limited in all sort of silly ways and attempt to use it as a general purpose language. It's not that you can't, but if you need say typing that can tell you if something is an array, then maybe picking JavaScript to begin with wasn't the brightest idea.
There's a ton of libraries and hacks out there, all attempting to fix the fact that JavaScript isn't really good general purpose language. ES6 is fixing a lot of these thing, but it's a little late in the game.
That's true. However:
Every dependency you add to your project is also a commitment.
When you add a dependency, you're committing to deal with the fallout if the library you're pulling in gets stale, or gets taken over by an incompetent dev, or conflicts with something else you're using, or just plain disappears. If you add a dependency for just a few lines of code, you're making a way bigger commitment than if you'd just copy/pasted the code and maintained it yourself. That's why so many people are shaking our heads at a 17-line dependency. It's way more risk than it's worth. If you need a better stdlib for your language (some of us write PHP and feel your pain) then find one library that fills in the gaps and use that.