Issue With Watching File Changes in Docker

Days ago, I was struggling to get live reloading/hot reloading (HMR) working for the code mounted in a docker volume on Windows. Volumes in docker serve the purpose of persistence — which means you can sync the container data with your local data. That works really well, but the catch comes when you want real time sync between both the locations of code — your local filesystem and docker's filesystem.

The Problem ^#

Different operating systems have different implementations of handling file events, example MacOS has the FSEvents API, linux has something known as inotify and Windows has the FileSystemWatcher.

Docker uses it's own filesystem and it's not necessary that your system's filesystem matches docker's. Thus, the file event handling APIs available on your system might not be available in a docker container. For example, the FSEvents API of MacOS is not available in a linux environment. Now, maybe I am missing some details here but, this might cause some discrepancies in how those file events are handled — if they are even handled at all. This can cause hot/live reloading to not work at all if you update your local code mounted on a volume.

I noticed this issue on Windows, so not sure about how this should be affecting other platforms.

The Fix ^#

The fix to watching file changes in docker is to use polling. Polling is a way to periodically check for changes that may have taken place. In polling you don't get notified, you keep checking the state over a network. Over the network approach works for docker because now you don't have to deal with the file system notifications, you can listen on the changes over the network.

If you are using a bundler such as webpack or any other developer tool such as gulp, browser-sync or livereload, they all use a cross platform file watcher named chokidar. Chokidar relies on the nodejs's file system API but it improves upon the nodejs' API. Chokidar supports polling and you can also set the polling interval.

Here's an example of how you can use polling when using gulp:

gulp.watch("<path>", {
    interval: 1000,
    usePolling: true,
}, task);

Drawback Of Polling ^#

The only reason polling is bad is that it's slow and consumes lot of CPU resources. If you have too many source files to watch for and you don't want to allocate more space, then polling is not a way forward. Quoting a wonderful analogy about polling from Raymond Chen:

"It’s like checking your watch every minute to see if it’s 3 o’clock yet instead of just setting an alarm." - Raymond Chen

Learn more about the performance consequences of polling on "The Old New Thing" — written by Raymond Chen.

An Alternative — Docker Compose Watch ^#

If you want to avoid polling, docker provides a way to watch file changes with the help of 'docker compose watch'. I removed volumes in lieu of this new watch option which is available in docker compose version 2.22 and later.

# docker-compose.yaml
services:
    your_service_name:
        build:
            context: .
            dockerfile: Dockerfile
        env_file:
            - ./.env # path to env file
        command: "npm run dev"
        develop:
            watch:
                - action: sync # 'build' is another action type
                  path: ./src # host directory path
                  target: /app/src # container directory path to map
                  ignore:
                    - node_modules/
        ports:
            - 127.0.0.1:3000:3000

However, you also need to create a USER which can edit files inside the container. The best practice is to create a non-privileged user and assigning that user as an owner of those copied files.

# Dockerfile
FROM node:20-alpine
RUN apk add --no-cache shadow

RUN useradd -ms /bin/sh -u 1001 app
USER app

COPY --chown=app:app . /app
WORKDIR /app

RUN npm install

For alpine linux, to be able to use useradd, you need to install the shadow package in your image using RUN apk add --no-cache shadow.

This approach works but personally I didn't find any significant performance improvement over polling. At the time of this writing, docker compose watch functionality has a bug which occurs when you terminate the watch command. You can't run the watch command again due to this bug. Follow the steps mentioned in this github issue comment to get the watch command running again.

Conclusion

I don't know how docker compose watch works under the hood and does it use polling as well, but it's clear that now you have two approaches to get file watch working in a docker container. A slight advantage of using docker compose watch is that you can also add an action named build which will rebuild the image and container. For example, you can rebuild the image on the package.json file change.