I know. I know. I’m a bit late to the game – and this was long overdue…
For a few years now, I’ve been fully submerged in the world of Enterprise Angular development – which has unfortunately led to me falling behind on some technologies, such as Docker. While I’ve been aware of LXCs for many, many years now, I’d not gotten past 101 setup tutorials for Docker to date – which changes today.
This post details my initial journey with running my first Docker container based Node.js application under Windows 10 – as well as the challenges encountered with live reload via nodemon during the journey, and how I got everything working well enough in the end.
As is often the case, I went looking for a recent basic getting started writeup, which led me to Dinushanka Ramawickrama’s “Let’s Dockerize a Nodejs Express API“. Being a huge fan of Node.js, often choosing it as my stack of preference, this seemed like a great place to start – taking a Node Express API and shoving it into a container using Docker.
All in all, I have to say that Dinushanka’s write up is pretty good – short, to the point, easy to follow, and accurate. It wasn’t until getting the last section, titled “Dynamically Change the Contents of the Running Container using Nodemon.”, that I ran into problems – for the life of me, I could not get live reload via nodemon to work as expected. Docker would output log messages seemingly indicating that all was well, but when I’d make changes to my server they weren’t syncing in my Docker container.
As I often do in situations like this, I turned to the comments – assuming that someone else had already experienced my pain and provided me a nicely packaged solution. Nope – not this time. So I went searching, which led me to a relatively recent issue on nodemon’sgithub repo: https://github.com/remy/nodemon/issues/1447 . As suggested by a commenter, I checked out “Application isn’t restarting” in the nodemon readme – https://github.com/remy/nodemon#application-isnt-restarting . Going for the low hanging fruit approach, I updated my package.json’s dev script to include the Legacy Watch flag:
Now, all that was left was to rebuild my image and rerun it:
> docker-compose down
> docker-compose build
> docker-compose up
With the above in place, all was well in the world – containerized live reloading node development. AIN’T LIFE GRAND?!
Long story short, essentially the mechanisms leveraged on Linux systems to detect filesystem changes aren’t available in Windows – and so live reload doesn’t work out of the box. Fortunately, nodemon’s Legacy Watch uses a different (though more costly) polling mechanism to detect changes – which works fine for our purposes of ‘getting started’. Note: this is actually a known issue, as you can see in the Docker official docs: https://docs.docker.com/docker-for-windows/troubleshoot/#inotify-on-shared-drives-does-not-work
So, you’ve recently bought an Ender 3, as I have. After reading up, you’ve decided to take the plunge and upgrade your firmware to Marlin. You open up your box as you’ve seen in countless YouTube videos – but something’s different from any of the videos that you’ve seen. Instead of 1.1.2 or 1.1.3, your board says “Creality3d V1 1.4” (1.1.4)!
If you find yourself frantically searching Google, only to find yourself coming up empty handed on instructional videos that explicitly mention v.1.1.4, I’m happy to announce that the firmware upgrade process is exactly as covered in the 1.1.2/1.1.3 videos and tutorials. I have now successfully completed the firmware update to Marlin TH3D_UFW_U1.R2.9a.
As all of the recommended tutorials cover, I had to first flash the bootloader using my UNO before being able to directly update the Ender 3’s firmware via USB. I’ve seen some Amazon sellers indicate that their v.1.1.4 board is already loaded with a bootloader, but I was unable to properly connect to the Ender 3 until I first loaded the bootloader – YMMV.
Installing Node.js in the Windows Linux Subsystem (WLS) is quick and easy – accomplished by essentially running 2 commands. By reading the official Node.js docs, we can see that Ubuntu installs are provided via NodeSource. Once at NodeSource, we see the commands required for Debian based Linux systems, such as Ubuntu (as of the time of this writing, I am running the Ubuntu 18.04.01 LTS subsystem).
First, we need to pull down the installer script and execute it using a 1-liner:
A bit about the command that we just ran. As you’re likely already aware, curl is a command line utility for interfacing with various protocols – in this case, http(s). If so inclined, you can take a look at the contents of the “setup_11.x” bash script. So, as you know, we’re working with a bash script – which we’re pulling down with the assistance of curl, but rather than saving it to our local filesystem, we’re piping it’s contents to bash, which we’re elevating using sudo. As you may have noticed, we’re actually calling “sudo -E”, which instructs sudo to preserve environmental variables while executing. So we call “bash -” as administrator, that training dash instructing bash “take the input from the pipe and treat the contents of a bash script”. By inspecting the output, or better yet from reading the bash script, we can see that we’re essentially modifying our apt sources to allow us to ‘apt install’ Node.js using the NodeSource repo. Easy peasy.
sudo apt-get install -y nodejs
With our apt sources updated to include the NodeSource repo, we use “apt-get install” to install nodejs – just as we would any apt package. Just as is the case with other apt installed packages, apt will evaluate all required dependencies for Node.js and install them on your system as needed before installing the Node.js binary.
Once installation has completed, you should be able to execute “node -v” and “npm -v” to see your installed versions.
We now have a fully functional Node.js environment in our Windows Linux Subsystem!
I have recently been building an API using NestJS (Angular-like dev flow for writing Expressjs apps in TypeScript). While developing under Windows is pleasant enough, I was somewhat surprised at how painful deploying the application to IIS via IISNode can be. I quickly discovered that Nodejs applications often don’t behave identically when running under IISNode. I also quickly discovered that debugging an application running IISNode isn’t particularly straight forward in 2018. If you’ve found yourself reading this post, you’ve likely discovered as I have – that IISNode support in 2018 can be difficult to come by, despite IISNode still being a cornerstone to Nodejs support on Windows.
As I hit my first obstacles with IISNode, I of course fired up Google and started looking for guidance. While I found a number of posts and writeups detailing how to debug an IISNode hosted app, most of what I ran across was dated and seemingly no longer works. If you find yourself going down this rabbit’s hole, you will quickly find many threads on StackOverflow and github without any real resolution. I really started to sweat when I ran across the official original IISNode github repo – which has largely been abandoned for a number of years. (At least now the repo points you towards the slightly better maintained Azure fork due to a little Twitter exchange).
Since things do behave differently at times under IISNode, the ability to debug is crucial – and without it, I found myself debating rewriting my entire API to a stack more recently supported for IIS debugging. But before throwing the baby out with the bathwater, I figured I would try a couple more things to see if I could salvage my API codebase.
Since the documented debugging approach no longer seems to work, and since there are so many unanswered problems regarding this on StackOverflow and GitHub, I figured my time and energy was better served elsewhere. While perhaps a solution does exist to get the documented debugging approach working, many who have come before me have seemingly failed – and ultimately abandoned using IISNode as a result. I don’t want this to be me.
Developing a NodeJS application on Windows in VSCode is a very pleasant experience when running under NodeJS directly. Node module support in Windows by popular libraries has come a long way over the last several years – and so most things just work. Knowing that we already know and like the VSCode approach, I figured this was a good starting point – “get VSCode setup to debug an IISNode hosted application somehow”.
In reviewing the VSCode node debugging info, I ran across “Attach to Remote”. While my intent was to debug the staging environment locally in VSCode (vs remotely), I figured this was an approach that I’d not yet tried and so was at least worth a glance. I further assumed that VSCode wouldn’t particularly care if I was attaching to some remote IP or a local one, so long as I was checking off all of the required boxes.
By default, VSCode doesn’t have any launch configurations setup for our project. Since there are no default configured launch profiles, you should find a screen similar to the following when you click on the ‘debugging’ icon (4th icon on the left menu). To get started, we need to create a launch.json, which will populate the debug dropdown currently displaying ‘No Configuration’.
We can manually create the “.vscode/launch.json” file in the root of our project if we desire, but I’m going to let VSCode do that for me. Let’s start by clicking on the the dropdown that currently reads “No Configurations” – and then selecting “Add Configuration”, before finally selecting “Node.js” as our environment.
You should now be staring at the generated launch.json file, which most likely looks like the following screenshot.
Since we’re just here to get debugging working, we’re just going to edit this configuration rule rather than creating a new one. We can begin by changing the value for “request” to “attach”. Let’s go ahead and update our name to something more accurate too – “Debug IISNode”. We can delete the “program” setting altogether. Now we need to add a few more settings – namely “address”, “port”, “localRoot” and “remoteRoot”. Some of these values will vary based on your local environment – see the screenshot below for mine.
As you may have noticed when reading my “localRoot” and “remoteRoot” values, I am simply debugging against the express example that’s installed by the IISNode Full installer. You can debug any Nodejs application that you desire by updating these paths accordingly.
Since the Express example application doesn’t have a iisnode.yml, I’ve copied the one included in the ‘configuration’ example that was also installed by the IISNode Full installer. We will need to make a few modifications to the copied iisnode.yml to get things working like we want. First, we’re going to need to uncomment the “nodeProcessCommandLine” setting and ensure that the path is the correct node.exe path – we’re also going to want to add ‘–inspect’ following the closing quotation as you see in the screenshot below, as this is the glue that makes debugging possible.
Another change to the iisnode.yml setup that you will likely want to make is to make sure that ‘nodeProcessCountPerApplication’ is set to 1. By default, IISNode will spin up 1 node.exe instance per processor that you have on the machine – and so you may find some unnecessarily complications when debugging against 4 instances of your application at the same time.
While I am uncertain if this helps anything, I also like to update my iisnode.yml to disable the build in debugging which no longer seems to work in 2018. This is optional, and I have found that our VSCode debugging will work regardless – I just like to do this to eliminate any possible unwanted magic. I also, at least initially, like to enable logging in my iisnode.yml, as it can help identify startup issues happening before your logic is up and running (and hopefully logging).
With everything saved, it’s a good idea to restart out website in IIS. Truth told, this shouldn’t be required as changes to the web.config and iisnode.yml should trigger a reload, though I’ve previously ran into issues that I believe perhaps were related to not all changes being properly loaded, so I now just do this explicit restart for good measure.
Now, using a tool like Process Explorer, we can inspect the node.exe instance that IISNode is using and see that it’s been launched with the “–inspect” flag.
Now we just have to launch our new debugging config. Once it’s selected from the dropdown, we can either click the ‘Play’ button or press F5. Since we’re trying to debug, it’s probably a good idea to set a breakpoint or two.
With the debugger attached, we can now visit our running application – once we hit a route associated with our breakpoints, we will find that we now have a fully functional debugging environment in VSCode for our IISNode handled application!
I didn’t see this documented elsewhere, and so figuring this out was a bit of a challenge – hoping that this saves a little bit of headache and heartburn!
Howdy again, lone reader (okay.. there’s actually 2 of you…)!
I’ve recently continued to work with Angular 6 and have really enjoyed the updated experience! As mentioned in the last post, I’d recently been presented with the challenge of finding a way to secure the implementation details of a library in a way that continued to support JIT and AOT builds.
As previously discussed, while some success was found with Angular 4 using Rollup, I could never get a single package capable of JIT and AOT – and instead had to rely on a process of generating 2 packages (one for our builds and one for 3rd party devs). And as previously discussed, I’ve found this process to be much more streamlined in Angular 6 due to the first class library generation support that Angular 6 brought to the table via ng-packagr.
I am an an enterprise architect who currently calls the arena of Fintech home. Years ago, we set out to modernize our main client facing systems and their supporting APIs. For the front end, we set our targets and AngularJS and didn’t look back. Our challenge wasn’t a small one – coming from a .Net and Java heavy world in terms of our existing code bases, part of our systems included a 3rd party ecosystem which allowed partner developers to extend our base functionality. Not only could partner developers extend the system’s core functionality, but they could also package up their extensions in a manner compatible with our systems so that they could be resold to other organizations who also benefited from this new functionality.
The initial release of our revamped, albiet feature-limited system, was very well recieved – and almost immediately the ecosystem began to grown around it at a pace nearly exceeding our ability to continue to build out our initial system… A good problem to have – and so we continued to iterate and release.
Fast foward a year+ and several iterations later, we found ourselves with a functional codebase now ported to Angular 4. Angular 4 provided us some new and interesting challenges – namely that the ballgame, and its associated rules which we had build our system around had changed. While we had completely redesigned our extensibility approach, which is outside of the scope of this document, we were faced with the very real problem of the 3rd party developer flow, which had seemingly degraded from Angular 1.x to 4.x under our design.. While I can’t go too far into the details, a lot of the 3rd party work had to be done out of band – integration testing with the core platform required round trips to the build server under our design just to get a working environment to test your enhancements (which may have been broken due to an errant keystroke that had gone unnoticed before pushing for the build – so you had to do it all over again).
This approach ‘worked’, but was never quite favored – it felt hacky and was difficult to maintain multiple builds…
Needless to say, we had to find a better way.
A Better Way
Enter Angular 6 – with it’s fancy first class support of library generation. Not quite knowing what we were getting into, a port of our 4.x to 6.x was undertaken – and went surprisingly well. From here we worked to break out all of our implementation which was intended to be ‘protected’ in to Angular Libraries (with the help of the Angular CLI and ng-packagr). Once our cleanup efforts were complete, we had a handful of Angular 6 feature libraries and what is essentially a shell application to tie them together.
While I am still struggling to grok the hows and whys that this works, it seemingly just does for now. In Angular 4 Land, we could never quite get our obfuscated/minified package to work for AOT Prod builds, though it worked perfectly fine for JIT builds; hence our need for 2 pacakges. But I am happy to report that the rules are apparently different in Angular 6 Land – as we are now able to create this ‘one package to rule them all’ in a relatively straight forward manner.
During some of the exploratory phase, I’d spoken on gitter with Alex Rickabaugh (@alxhub) about what we were trying to accomplish. While our interactions were small in regards to our Angular 4 efforts, I popped back in to update Alex on the success we’d found with Angular 6 – at which time he requested that I do a quick writeup in the case that it might assist others….
…so here we are…
Here be dragons…
As previously mentioned, the steps are currently pretty straight forward. For the following overview, let’s assume we arrived here from “ng g library awesomesauce”.
– “ng build awesomesauce –prod”
– Navigate to /dist/awesomesauce/
– Delete the esm5, esm2015, esm5, and fesm2015 folders.
– Navigate to /dist/bundles/ and delete the non *.min.* resources and the map associated with your minified bundle.
– Back in /dist/awesomesauce/, open up package.json and make the following edits:
– update ‘main’ to reference the *.min.js bundle in /bundles/
– Delete ‘module’, ‘es2015’, ‘esm5’, ‘esm2015’, ‘fesm5’ and ‘fesm2015’ key/value pairs
– “ng pack”
Note: Again, here be dragons. I am not a web security expert. The process described here may or may not stand up to rigorous scrutiny. We’re in the process of vetting this and working to clear the security requirements imposed upon us – but so far this appears to be fitting the bill for us, so perhaps it will for you too.
So I decided to make things permanent! immatt.com is now the official home of Matt Ezell! After years of living at mattezell.info, I’ve decided to make the plunge and go TOP LEVEL! WHAAAA?!?!
While I still hold mattezell.info, I will be redirecting its properties to immatt.com going forward. All new projects coming out of the Matt Brand™ will likely now live somewhere in the vicinity of immatt.com unless otherwise branded.
For those wishing to trip down memory lane, mattezell.info’s original content now lives at archive.immatt.com – I just couldn’t bear to part with it, so there it is… The new blog of course lives here, at blog.immatt.com. I’m still up in the air as to what I am going to put at the root immatt.com, so for now it’s 302 forwarding to here (blog.immatt.com).
Anywhos… Welcome! And tune back in soon to find out what comes next!