Who Does the Web Server Belong To?

While recently reviewing the web servers of my projects, I finally began to wonder about the different ways each project manages its web server. I currently handle several services, but in one, the web server configuration and deployment process are included within the frontend project, while in another, the web server is managed within the server project. Curious about this difference in management, I asked around at other teams and acquaintances at other companies, but I found that management approaches largely split along the two forms above. I think these approaches can vary depending on each environment, but I want to look into web servers and organize my thoughts on how best to manage the web server for each project.

What Is a Web Server?

A “web server” could simply be described as a server that sends back an HTTP response for an HTTP request from a browser. According to MDN's “What is a web server?”, a web server can be broadly divided into hardware and software aspects, and into static web servers and dynamic web servers.

Since the dynamic web server mentioned above is generally considered closer to an application server, here I plan to look into web servers based on the software-side static web server.

Static Web Server

A static web server is a server that sends unchanging static resources such as HTML, CSS, JS, and image files as HTTP responses matching the browser's HTTP request. Here, the web server must find the stored files at the designated path and respond to the request without any additional processing.

Representative web servers include nginx and Apache; nginx has been used since 2004 and Apache since 1995 by countless companies. Thanks to the trust built up over years in handling large-scale traffic, compatibility with operating systems, browsers, and CDNs, and security, I think they're still the main choices for a web server.

Setting Up a Web Server

I'll walk through setting up a web server with nginx in a Linux environment. Linux is widely used for building an nginx web server because, it's said, the OS offers excellent stability and performance and lets you actively leverage high-performance networking in the system. In the case of Windows, nginx has many feature limitations and Linux is also said to be superior in performance. Additionally, the Windows version of nginx is considered a beta version.

nginx

In a Linux environment, checking for and installing nginx can be done simply with the commands below.

cli
$ nginx -v

# update the package list
$ sudo apt update

# install nginx
$ sudo apt install nginx -y

Below I've summarized the basic settings used in nginx. These settings are defined in a file called nginx.conf and are used when the nginx web server runs.

conf
# nginx main configuration file

# the user to run the nginx process as
user  www-data;

# number of worker processes (auto: set automatically to the number of CPU cores)
worker_processes  auto;

# error log file path and log level (record only warn and above)
error_log  /var/log/nginx/error.log warn;

# location of the nginx process ID file
pid        /var/run/nginx.pid;

# -----------------------------
# event processing configuration block
events {
    # maximum number of connections each worker process can handle simultaneously
    worker_connections 1024;
}

# -----------------------------
# global HTTP configuration block
http {
    # include the MIME type definition file (used for the Content-Type header)
    include       /etc/nginx/mime.types;

    # default type to use when the MIME type is unknown
    default_type  application/octet-stream;

    # access log file path setting
    access_log  /var/log/nginx/access.log;

    # use the sendfile system call when transferring files (performance improvement)
    sendfile        on;

    # connection keep time (seconds) - keep-alive setting
    keepalive_timeout  65;

    # -----------------------------
    # Gzip compression settings (optional performance optimization)

    # whether to use Gzip compression
    gzip on;

    # list of content types to compress (text, JSON, JS, etc.)
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    # -----------------------------
    # default server block settings (virtual host)

    server {
        # the port this server block listens on (HTTP default port 80)
        listen       80;

        # the domain name to accept (here, handle only localhost)
        server_name  localhost;

        # web root directory (where static files are located)
        root   /usr/share/nginx/html;

        # list of index files to load by default
        index  index.html index.htm;

        # -----------------------------
        # request routing settings

        location / {
            # if the request path is an actual file, serve it; otherwise return 404
            try_files $uri $uri/ =404;
        }

        # -----------------------------
        # error page settings

        # custom page to show when a 404 error occurs
        error_page 404 /404.html;

        # allow only internal handling of /404.HTML requests (block external access)
        location = /404.html {
            internal;
        }
    }
}

CLI

Running nginx in a Linux environment can be done, along with checking status, via the system commands below. In a Docker container environment, however, since systemd (the init system) isn't included, you have to run the nginx process directly or control it manually.

cli
# run system commands in a Linux environment
$ sudo systemctl start nginx
$ sudo systemctl enable nginx
$ sudo systemctl status nginx

Dockerfile

In a Docker container environment, you have to run it in foreground mode when the container starts via the CMD setting in the Dockerfile. nginx runs in the background by default, and it's said that if you don't run it in the foreground in a Docker container environment, it terminates immediately.

• Foreground: a setting that runs the process without detaching it, occupying the terminal and continuously retaining control

dockerfile
FROM nginx:alpine                          # lightweight base nginx image based on alpine Linux
COPY ./html /usr/share/nginx/html          # static file location
COPY conf/nginx.conf /etc/nginx/nginx.conf # overwrite the nginx config file -> custom server settings
CMD ["nginx", "-g", "daemon off;"]         # run nginx in the foreground

After running nginx in the form above, when a browser accesses the web server's domain, the DNS server converts the domain to the matching IP and the request is sent, and the web server processes the response matching the request. Let me summarize how we've been managing this resource-handling process.

Static Resources

Static resources broadly include HTML, bundled JS, images, fonts, and so on. In this explanation of static resource management, I won't cover things like how the script bundle is built, how images and fonts are optimized, or how resources are loaded within the HTML. I'll focus on this post's main topic: where and how the web server serves static resources.

CDN

I'd guess most static resources are managed via a CDN (Content Delivery Network). The web server that first receives the request to the domain responds with the HTML located in the same container. Then the browser receives the HTML response and requests the resources matching the CDN paths, receiving them in response.

• Browser → Web Server → HTML response
• Browser parses HTML → CDN → resource response

I think the processing above is probably the normal response-handling structure between the web server and CDN as I picture it. However, a particular project I manage handles resources in a different form.

To start, all resources including the HTML are managed on the CDN. Since the HTML is managed on the CDN, when the browser first requests the domain, the web server proxies to the CDN and responds by serving the HTML. And other static resources are also not requested directly from the CDN but requested to the web server, which, like the HTML, proxies to the CDN and serves the resources.

• Browser → Web Server → CDN → HTML response
• Browser parses HTML → Web Server → CDN → resource response

As for why this structure came about, I was able to find what I think is the reason for the current management method by asking the people who actually built the project at the time. It could be seen as a problem that can arise when the web server is managed not by the frontend but by the backend, or when ownership of management is unclear. In the structure above, all static resources including the HTML are managed by the frontend, so the frontend can be said to have fulfilled its entire role by uploading the build artifacts to the CDN. The backend can be said to have done its part simply by running a web server with the static resource paths designated; and since management of the HTML is handled by the frontend, it can be said the backend fulfilled its role by designating the CDN path for the HTML on the web server. As for why the paths to resources within the HTML were designated to the web server rather than to the CDN, I was told it was, in that work situation, presumably simply to unify the resource request paths within the frontend.

In fact, even with the handling above a normal service is possible, but let me summarize the parts I thought were problems. First, the HTML should always be served as the latest deployed version, but if an inappropriate cache invalidation strategy is used in the process of going through the web server to fetch from the CDN, incorrect HTML could be delivered to the user. And when resources including HTML requests are served from the CDN through the nginx proxy, it would be processed more slowly than receiving the response directly from the CDN.

Docker Container

Among the projects I currently manage, there are also projects that don't use a CDN and instead place the resources inside the same Docker container as the web server, with the web server serving those resources directly.

Below is a Dockerfile example to simply illustrate the structure above.

dockerfile
# Stage 1: build stage (build the React app with Node.js)
FROM node:18-alpine AS build

WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build

# Stage 2: run stage (serve static files with nginx)
FROM nginx:1.21-alpine AS run

# copy the built files to nginx's default directory
COPY --from=build /app/dist /usr/share/nginx/html

# copy the custom nginx config
COPY nginx.conf /etc/nginx/nginx.conf

# expose port 80
EXPOSE 80

# run nginx (foreground mode)
CMD ["nginx", "-g", "daemon off;"]

When managing a service with a structure that includes static resources in the Docker container, you can manage the web service with a single container, allowing management with a simple build/deploy structure. And since there's no dependency on a CDN, normal service is possible even during a CDN outage. However, since the static resources are included in the image, the image size increases, and deployment time and storage costs can rise. Also, for services with heavy traffic, the network costs and scaling costs for static resources can increase exponentially.

The project with this structure was decided during the initial project design when, at that time, CDN outages occasionally occurred, based on the server side's opinion that it would be better to manage things internally within the service so as not to be affected by CDN outages and provide normal service. In this project, the frontend project's build/deploy process needs to include the static resources in the image. So the nginx config and a Dockerfile that builds and copies the static resources and runs the nginx web server, as in the example above, are managed within the frontend project. And the frontend's role is completed up to build/deploy, after which the Docker container for the web server is managed by the server side.

Who Does the Web Server Belong To?

Looking back now, I don't think I ever considered the web server to be something under our management. As a result, I only belatedly learned about odd structures like the one above, and even in situations where they need to be resolved, I think discussion with the server engineers is necessary. In fact, there are projects where the server engineers manage the nginx configuration or the running server, so this could be considered a given. Of course, when needed I do check the nginx setup and configuration and manage fixes when there are problems, but for the sake of a better web service, I think it would be good for the frontend developer to be clearly assigned as the owner and manage everything from the web server configuration to the entire environment.

Recently, looking at the frontend requirements at various companies, some include web server management or k8s operation experience among the preferred qualifications, and occasionally include them in the required qualifications. Seeing situations like these, I think the era has already arrived where we must be able to move beyond frontend developers who simply build web pages and become developers equipped with capabilities that include the web server and infrastructure.

"The best way to prepare for the future is to shape it." - Peter Drucker -