Code Buckets

Buckets of code

Node

Npm link set up advice and troubleshooting

Working with npm link wasn't as straightforward as he'd hoped.
Working with npm link wasn’t as straightforward as he’d hoped.

I’ve been fighting against npm link recently and I’ve a funny feeling I’m not the only one. I don’t find the documentation particularly useful, the way it works seems opaque bordering on mystical and the cli output is in cryptic riddles.

So, in the interest of alleviating npm related suffering, here’s some tricks and tips to get to npm link working, to troubleshoot it when it doesn’t work and a few alternatives if you need them.

Npm link is used when you want to work against multiple repositories at the same time. The typical use case is when you are developing an npm package. Rather than publishing your package to test it, you would use npm link to consume it from a test project.

When referencing an npm package, normally you would pull a copy of it down from a remote source e.g. npm.org

When you use npm link, it sets up a symlink in your global node_modules folder which points back to your local environment. Therefore, you can reference your package on your machine as if it were published on npm.

To demonstrate, I’m using a quick demo application which gives news, travel and weather updates for Brisbane. It’s available on my GitHub site at

https://github.com/timbrownls20/Demo/tree/master/Node/npm-link

The application serves out a web page which gives a useful update on how things are going currently in Brisbane. If I run the application I can clearly see that the news is that ‘everyone’s awesome’ and the weather is ‘hot and sunny’

News from Brisbane

If I refresh the web page I get a useful update that the weather is now ‘hot, hot, hot’ and that I should not have any further worries about the news. Also, the traffic has changed from ‘Steady Away’ to ‘Open Roads’ which is important information for all commuters.

Important Brisbane news update

The application is a Node app with a master project using express which consumes 3 npm packages, one each for news, travel and weather. 

However, when I want to change the underlying packages,I will need to have everything installed locally – master project and packages. Therefore, I need to use npm link.

To set up, run npm link for each package project and the master project, four projects in all. The commands in order are…

Link news package

cd ./brisbane-news
npm link

Link travel package

cd ../brisbane-travel
npm link

Link weather package

cd ../brisbane-weather
npm link

Link news travel and weather to the master project

cd ../master-project
npm link brisbane-news brisbane-travel brisbane-weather

The master projects will now be referencing the local versions of the packages and not the npm ones. 

Unfortunately, the command line tells us very little about what has or hasn’t happened. How can I tell if it has worked and we are using the local repositories, not the npm ones?

It’s non-obvious to know if npm has worked and that you are referencing the local version of your packages. Below are a few ways to find out.

1. Navigate to node_modules and look for changes

Use your IDE navigator and browse node_modules to the local packages. If it has worked, you will be redirected to your local copy. So browsing node_modules and looking at the brisbane-news package you can see the local changes in your IDE.

Local code changes if browsing to the package under node_modules.

In a similar vein, if you navigate to the file system directly from the node module selection it will be very obvious if you are in your local dev copy or marooned in the middle of node_modules and still referencing the remote copy.

Selecting ‘reveal in folder’ should go to the local copy of brisbane-new if npm link has worked. If it hasn’t ,then you will still be in the node_modules folder

It is a pain to navigate your way around node_modules to find this out, but it’s a definitive answer if you do.

2. Your IDE might tell you

VS Code flags packages that are referenced through npm link with a little arrow.

Little arrows next to the packages indicate they are npm linked. Sometimes works.

It’s kind of nice but kind of useless at the same time. The little arrow doesn’t always show, even when you have definitely linked to your local copy. Still, if you can see the little arrow it’s encouraging.

Also, if you hover over the import statement in VS Code then it will pop up the actual path it’s resolving to, so you can see at a glance if it is going to the right place. This doesn’t work with require statements though.

Obviously other IDEs will have different ways to indicate if npm link is working so, it’s worth finding out how your editor might be able to help you.

3. List your global packages

Since npm link works by installing symlinks in the global node_modules, then examining what is actually globally installed is illuminating. Run …

npm ls -g

To get your currently installed global packages. The output shows any global packages that are linked to a local path i.e. they are npm linked

output for npm ls -g

So here I can see clearly that brisbane-new, brisbane-travel and brisbane-weather are definitely local – they have a local path.

4. Navigate to global node_modules folder

It’s probably not something you’ll often reach for, but it’s instructive to navigate to your global node_module folder on the file system and see what is actually there. 

My global node_modules as shown on a Mac. Alias folders are symlinks

In my folder, I can clear see which packages are actually installed and which are symlinks back to another location on my dev machine. It really hammers home what npm link is doing.

Unfortunately when npm link doesn’t work it can be difficult to work out why. Here are a few things that might have gone wrong

1. Check you’ve linked every package

If you are working on multiple packages then you need to link them all in the same command. If you run multiple commands then you will only be linked to the last one. So running

npm link brisbane-news
npm link brisbane-travel
npm link brisbane-weather

Will only link the last package i.e. brisbane-weather. Running npm link unhelpfully deletes all the previous linkages. To link to multiple packages you’ll need to combine them into the same command i.e.

npm brisbane-news brisbane-travel brisbane-weather

This will now link all three packages.

2. Don’t globally install your local packages

If you accidentally install the package you want npm linked into your global module_folder, then it will override the symlink and you won’t be developing locally any more. If you run 

npm ls -g

You will see that the package is not local and you aren’t in Kanas anymore.

brisbane-news has no local path so is actually globally installed. This isnt what I want so I’ll have to uninstall it globally and relink

It’s easy to do this. It’s always worth checking your global node_module folder to see what rubbish you have inadvertently installed and taking out the trash.

3. Check you’ve linked both ends

You need to link the packages and the master projects by running npm link at both ends i.e in every project. It’s easy to forget and wonder why things aren’t linking up correctly.

4. Check you are on the right version of node

Because npm link works by installing links into your global node_modules, if you change version of node then they will all disappear. This often happens when swapping between versions using nvm (node version manager). Make sure you are using the same version of node that you used to set up the links.

5. Trace dependencies

In essence, npm link is an exercise in module resolution. If you can’t understand why your application can’t find your linked module then put a trace on it. If you are using TypeScript then compile with the traceResolution flag

tsc —traceResolution

And if you aren’t using TypeScript, then there are libraries to help you trace e.g. 

https://www.npmjs.com/package/trace-deps

6. Delete node_modules

This really is like ‘turning it on and off again’ but it’s always worth deleting all of node_modules and reinstalling so 

rm -rf node_modules
npm install

Then relink the projects with npm link. A bit more subtly you could just delete the npm package you are trying to link and then relink. Never give up without trying this.

Alternatives

But if you do give up on npm link then, there are a number of alternative solutions 

1. Monorepo plus GitHub submodules

You could go for the fashionable option and move towards a mono-repo solution. If you put your module as a git sub module in the monorepo, then it can have a completely separate repository but build out as part of the entire solution using the monrepo magic. No npm link needed as it’s all one.

2. pnpm

I’ve only got word of mouth for this, but I keep getting told that pnpm is a better option as a package manager than npm. It’s a drop in solution so really easy to swap over. If you think it’s npm rubbishness that is stopping npm link working then it might be worth trying an alternative package manager.

I’ve put a task on the backlog at work to swap from npm to pnpm, so I’ll have a better idea in a few weeks whether pnpm is all it’s cracked up to be.

3. Publish it

If all else fails, then you can always publish the solution and test it as a normal npm package. I’ve done this before and I’m not proud of it… but it is guaranteed to work. And if it doesn’t work then you know you’ve got bigger problems than npm pickiness.

https://docs.npmjs.com/cli/v8/commands/npm-link
Npm link offical documentation

https://superuser.com/questions/253935/what-is-the-difference-between-symbolic-link-and-shortcut
What symbolic links are and why they are not the same as windows shortcuts.

https://www.typescriptlang.org/docs/handbook/module-resolution.html
Good explanation about module resolution

https://stackoverflow.com/questions/5926672/where-does-npm-install-packages
It’s useful to poke around the actual global node_module directory sometimes. I like to find it, if I’m having npm link problems to prove to myself things are actually there and I haven’t gone mad. This link tells you where node installs its global files so you can do the same. It will be different if you have nvm installed. In that case find the nvm install location and look under versions.

https://turborepo.org/
If you are interested in a monorepo solution, then turborepo is a good option and the one I’m currently using. It’s recently gone open source and works well.

1 COMMENTS

LEAVE A RESPONSE

Your email address will not be published. Required fields are marked *