4

I'm planning to use the distroless base image to run container applications.

Since I'm using envoy in the k8s sidecar, when I stop the application container, I use the sleep command in preStop before safely stopping the pod.

lifecycle
 preStop:
  exec:
    command:
     - sleep
     - "40

Since the distroless image does not allow the use of shells, etc., we use a multi-stage build so that we can use the sleep command.

This simple golang application.

FROM alpine:3.13 as builder

ENV GOROOT /go

FROM gcr.io/distroless/static-debian10

WORKDIR /app
COPY ./dist/app ./
COPY --from=builder /bin/sleep /bin/sleep


CMD ["./app"]

The sleep command was failing when I started the Pod in this image and had it switch to another Pod.

Exec lifecycle hook ([sleep 40]) for Container

Is it not possible to do it this way? If I'm doing this wrong, I'd like pointers.

Thanks.

0

5 Answers 5

3

The "distoless" images really only make sense when your image consists of only the application you're trying to run, and absolutely nothing else. This is often a good practice, but trying to bring in the additional sleep(1) command isn't quite compatible with it.

I can suggest a couple of approaches to get around this:

  1. Figure out how to not need sleep(1) at all. "We need to wait 40 seconds, because Envoy" seems like you're working around a bug; fix that bug, instead of trying to delay the container shutdown.

  2. Reimplement the delay in the same language as your main application. In Python, for example, it's a 3-line script. Then everything you need to run it is in the image already.

  3. Get a statically-compiled sleep(1) binary. The easiest way to get this is from the BusyBox tool suite, which contains many common Unix utilities in a single binary; but Alpine Linux is basically just BusyBox plus a package manager, so in turn the easiest way to get BusyBox is to change your image to be FROM an Alpine-based image.

  4. Get all of the dynamic libraries the sleep(1) binary requires. You can run ldd sleep in an environment where it works to figure out what they are. You probably don't want to copy them directly from the host, though, and this is where normal Linux distributions come with these libraries prepackaged (and come with a /bin/sleep already).

In your Dockerfile you are probably getting a broken symlink /bin/sleep -> busybox. You could also copy the busybox binary over, but this essentially defeats the point of having a distroless image.

Sign up to request clarification or add additional context in comments.

Thanks Comment David !! Waiting for Envoy to finish is a common practice in applications using gRPC, so I don't see any problem with it. medium.com/incognia-tech/… I also tried multi-stage builds with Debian images, but it did not work. Symbolics don't seem to be broken to me.
for example, scrapbox.io/tkms0106/… I had imagined doing something like this.
1

In case someone else finds this post here. Since k8s 1.30 there is the PodLifecycleSleepAction . A new way to set a sleep in the preStop.

preStop:
  sleep:
    seconds: 10

This does the same but also easily works with distroless images.

Comments

0

see this

the default grace period for delete the PODs is 30 seconds, if one of the Pod's containers has defined a preStop hook, the kubelet runs that hook inside of the container. If the preStop hook is still running after the grace period expires, the kubelet requests a small, one-off grace period extension of 2 seconds.

Note: If the preStop hook needs longer to complete than the default grace period allows, you must modify terminationGracePeriodSeconds to suit this.

so I think try to set you sleep command to 20 seconds or try to set terminationGracePeriodSeconds to 60 seconds.

Tanks!! The advice you gave me was helpful in solving another problem.
0

Sleep command can now be used by using gcr.io/distroless/base-debian10 instead of gcr.io/distroless/static-debian10

This is not true. Sleep binary is not included in both base and static images, ref github.com/GoogleContainerTools/distroless/tree/main/base
0

I faced the same issue and here's a solution that helped me. Dockerfile:

FROM busybox:1.36-musl as builder

FROM gcr.io/distroless/static-debian11

COPY goappbinary /usr/local/bin/app

# We need sleep binary to execute preStop k8s lifecycle hook
COPY --from=builder /bin/sleep /usr/local/bin/sleep

k8s deployment:

...
  containers:
    - name: some-app
      ...
      lifecycle:
        # this is to prevent the pod from being killed while it is still serving requests
        # details: https://learnk8s.io/graceful-shutdown
        preStop:
          exec:
            command: [ "/usr/local/bin/sleep", "15" ]
      ...
      command: ["/usr/local/bin/app"]
      ...

Comments

Your Answer

Draft saved
Draft discarded

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.