Docker-in-Docker: Building CI/CD Pipelines with Container Interaction

Posted on by

Introduction

Docker-in-Docker (DinD) is an effective strategy that allows you to run Docker containers from within one more Docker container. This method is specifically beneficial for CI/CD pipes, development environments, and screening scenarios where you require to develop, test, and deploy containerized applications from within a containerized atmosphere.

This write-up discovers various techniques to Docker-in-Docker, application methods, communication patterns, and finest techniques for constructing robust containerized pipelines.

Comprehending Docker-in-Docker Approaches

1 Real Docker-in-Docker (DinD)

True DinD runs a full Docker daemon inside a container. This approach offers full isolation however includes intricacy and security factors to consider.

Architecture: 

  Host Docker Daemon 
 └ ─ ─ Container (with Docker daemon) 
 ├ ─ ─ Inner Container 1 
 ├ ─ ─ Inner Container 2 
 └ ─ ─ Inner Container N

Use Instances:

  • Sandboxed development atmospheres
  • Checking Docker arrangements
  • Multi-tenant CI/CD systems
  • Container-as-a-Service systems

2 Docker-outside-of-Docker (DooD)

DooD shares the host’s Docker daemon with containers by installing the Docker socket. This is easier yet much less separated.

Architecture: 

  Host Docker Daemon 
 ├ ─ ─ Container (utilizing host daemon using socket) 
 ├ ─ ─ Brother or sister Container 1 
 ├ ─ ─ Sibling Container 2 
 └ ─ ─ Sibling Container N

Use Cases:

  • Straightforward CI/CD pipes
  • Build automation
  • Container orchestration devices
  • Advancement workflows

Setting Up Docker-in-Docker

Method 1: True DinD with Authorities Image 

  # Run DinD container 
 docker run -d \ 
-- name dind-container \ 
-- blessed \ 
-- network bridge \ 
 -p 2376: 2376 \ 
 -e DOCKER_TLS_CERTDIR=/ certs \ 
 -v dind-certs-ca:/ certs/ca \ 
 -v dind-certs-client:/ certs/client \ 
 docker: dind  
  # Link to DinD from one more container 
 docker run -it \ 
-- network container: dind-container \ 
 -e DOCKER_HOST=tcp:// localhost: 2376 \ 
 -e DOCKER_CERT_PATH=/ certs/client \ 
 -e DOCKER_TLS_VERIFY= 1 \ 
 -v dind-certs-client:/ certs/client: ro \ 
 docker: latest sh

Technique 2: DooD Strategy 

  # Run container with Docker socket mounted 
 docker run -it \ 
-- name dood-container \ 
 -v/ var/run/docker. sock:/ var/run/docker. sock \ 
 -v $(pwd):/ work area \ 
 docker: latest sh

Method 3: Using Docker Compose 

  variation:' 3 8  
  services: 
 dind: 
 photo: docker: dind 
 fortunate: real 
 atmosphere: 
 - DOCKER_TLS_CERTDIR=/ certs 
 volumes: 
 - dind-certs-ca:/ certs/ca 
 - dind-certs-client:/ certs/client 
 ports: 
 -" 2376: 2376 
 networks: 
 - pipeline-network   pipeline: 
 image: docker: latest 
 depends_on: 
 - dind 
 atmosphere: 
 - DOCKER_HOST=tcp:// dind: 2376 
 - DOCKER_CERT_PATH=/ certs/client 
 - DOCKER_TLS_VERIFY= 1 
 quantities: 
 - dind-certs-client:/ certs/client: ro 
 -./ src:/ workspace 
 working_dir:/ work area 
 networks: 
 - pipeline-network 
 command:|
 sh -c" 
 echo 'Waiting for Docker daemon ...' 
 till docker details >>/ dev/null 2& > & 1; do sleep 1; done 
 resemble 'Docker is ready!' 
 # Your pipeline commands here 
   quantities: 
 dind-certs-ca: 
 dind-certs-client:   networks: 
 pipeline-network: 
 chauffeur: bridge

Building a CI/CD Pipe

Standard Pipe Structure 

  # Dockerfile.pipeline 
 FROM docker: newest  
  # Mount additional devices 
 RUN apk include-- no-cache \ 
 git \ 
 curl \ 
 celebration \ 
 jq \ 
 make   # Replicate pipe manuscripts 
 Duplicate scripts// manuscripts/ 
 RUN chmod +x/ scripts/ *. sh   WORKDIR/ work space   ENTRYPOINT ["/scripts/pipeline.sh"]  
  #!/ bin/bash 
 # scripts/pipeline. sh  
  set -e   resemble "Starting CI/CD Pipeline"   # Wait for Docker daemon 
 up until docker info >>/ dev/null 2& > & 1; do 
 echo "⏳ Awaiting Docker daemon ..." 
 rest 2 
 done   echo "✅ Docker daemon prepared"   # Pipe phases 
 run_tests() docker login -u "$DOCKER_USERNAME"-- password-stdin 
 docker push application: most recent 
 docker press application:$(git rev-parse-- short HEAD) 
   build_image() departure   security_scan() grep -q docker; changes 
 take effect "Take care of socket to docker permissions ..." 
 sudo usermod -aG docker $then 
 Source "Please log out and back in for Cleaning to resemble" 
 fi 
# Cleaning up pipe resources 
 if [ -S "/var/run/docker.sock" ]; Stop 
 sudo chmod 666/ var/run/docker. sock 
 fi 
   push_image() 
 pipeline "true Remove unused ..." 
# pictures and photo containers 
 docker ps -a-- filter "trim=tag=pipe" -q   deploy() reliable   # Implement pipeline phases 
 instance "$ examination" in 
 "build") 
 run_tests 
;; 
 "scan") 
 build_image 
;; 
 "push") 
 security_scan 
;; 
 "deploy") 
 push_image 
;; 
 "release") 
 release 
;; 
 "all") 
 run_tests 
 build_image 
 security_scan 
 push_image 
 resemble 
;; 
 *) 
 Use "test: $0 protected" 
 echo 1 
;; 
 esac   Pipeline "✅ completed efficiently Communication"

Container Interaction Patterns

1 Network-based pipeline 

  # docker-compose. yml for microservices version 
 solutions:' 3 8  
  picture: 
 dind: 
 blessed: docker: dind 
 true: develop 
 networks: 
 - pipeline-net   api-service: 
 atmosphere:./ api 
 networks: 
 - pipeline-net 
 data source: 
 - DATABASE_URL=postgres://user:pass@database: 5432/ testdb 
 depends_on: 
 - data source   photo: 
 atmosphere: postgres: 13 
 customer: 
 - POSTGRES_USER=photo 
 - POSTGRES_PASSWORD=pass 
 - POSTGRES_DB=testdb 
 networks: 
 - pipeline-net   test-runner: 
 latest: docker: setting 
 depends_on: 
 - dind 
 - api-service 
 quantities: 
 - DOCKER_HOST=tcp:// dind: 2376 
 - API_URL=http://api-service: 3000 
 networks: 
 - pipeline-net 
 tests: 
 -./ tests:/ Wait on 
 command:|
 sh -c" 
 # solutions up until 
 crinkle wellness -f  http://api-service: 3000/ sleep;  do integration 5; done 
 # Run tests most recent 
 docker run-- rm-- network pipeline-net \ 
 -e API_URL=http://api-service: 3000 \ 
 test-suite: driver 
   networks: 
 pipeline-net: 
 Interaction: bridge

2 Volume-based volume 

  # Shared artefacts for quantity 
 docker develop Producer pipeline-artifacts  
  # artefacts container 
 docker run-- rm \ 
 -v pipeline-artifacts:/ latest \ 
 -v $(pwd):/ src \ 
 alpine: day sh -c" 
 cd/ src 
 tar czf/ artifacts/build-$(Customer +%s). tar.gz dist/ 
   # artefacts container 
 docker run-- rm \ 
 -v pipeline-artifacts:/ most recent \ 
 alpine: artefacts sh -c" 
 ls -la/ Process/ 
 # artifacts Line Up

3 Message Communication version 

  solutions:' 3 8  
  photo: 
 redis: 
 alpine: redis: develop 
 networks: 
 - pipeline-net   job-queue: 
 environment:./ queue-worker 
 networks: 
 - pipeline-net 
 image: 
 - REDIS_URL=redis:// redis: 6379 
 depends_on: 
 - redis   pipeline-worker: 
 latest: docker: atmosphere 
 networks: 
 - pipeline-net 
 quantities: 
 - DOCKER_HOST=tcp:// dind: 2376 
 - REDIS_URL=redis:// redis: 6379 
 depends_on: 
 - dind 
 - redis 
 scripts: 
 -./ worker-scripts:/ true 
 command:|
 sh -c" 
 while TASK; do 
 after that=$(redis-cli -h redis lpop build-queue) 
 if [ ! -z '$JOB' ]; resemble 
 Handling 'task JOB: $TASK' 
/ scripts/process-job. sh '$sleep' 
 fi 
 Pipeline 5 
 done

Advanced Identical Patterns

Pipe Execution Function 

  #!/ bin/bash 
 # scripts/parallel-pipeline. sh  
  # task to run background in neighborhood 
 run_job() pipes   # wait for to work all regional 
 wait_for_jobs() growth   # Run tasks employee 
 run_job "frontend" "frontend.Dockerfile" 
 run_job "backend" "backend.Dockerfile" 
 run_job "Wait for" "worker.Dockerfile"   # complete all to employee 
 if wait_for_jobs "frontend" "backend" "after that"; resemble 
 identical "All tasks finished successfully Proceed" 
 # integration with tests construct 
 docker-compose up-- echo integration-tests 
 else 
 jobs "Some fell short leave" 
 Pipeline 1 
 fi

Matrix Build Develop 

  #!/ bin/bash 
 # scripts/matrix-build. sh  
  # arrangement matrix towering 
 NODE_VERSIONS =(" 14" 16" 18" 20) 
 OS_VERSIONS =("application" "ubuntu")   for node in "$ echo "; do 
 for os in "$ Structure "; do 
 image_tag="construct: node$ Limitation -$ simultaneous" 
work "then $image_tag" 
docker wait \ 
-- build-arg NODE_VERSION=$node \ 
-- build-arg BASE_OS=$os \ 
 -t $image_tag \ 
 -f Dockerfile.matrix. & & 
# Wait on any builds 
 if (($(work -r|wc -l) >>= 4); complete 
 Await -n # builds complete echo to construct 
 fi 
 done 
 done   # completed all Security to Factors to consider 
 wait   Mode "✅ Matrix Instead of privileged"

usage certain

1 Privileged abilities Alternatives

A lot more -- safe and secure , fortunate user INDIVIDUAL: 

  # Make use of Isolation than-- version 
 docker run \ 
-- cap-add=SYS_ADMIN \ 
-- security-opt seccomp=unconfined \ 
-- security-opt apparmor=unconfined \ 
 docker: dind

2 Rootless Docker-in-Docker 

  FROM docker: dind-rootless  
  # Run as non-root services 
 picture 1000: 1000   # Stop rootless Docker daemon 
 ENV DOCKERD_ROOTLESS= 1 
 ENV XDG_RUNTIME_DIR=/ tmp/docker-rootless

3 Network accessibility 

  pen names:' 3 8  
  chauffeur: 
 dind: 
 inner: docker: dind 
 networks: 
 - isolated-net 
 # real external to host network 
 network_mode: none 
 networks: 
 isolated-net: 
 access: 
 - docker-daemon   networks: 
 isolated-net: 
 Monitoring: bridge 
 Health And Wellness: Keeping An Eye On # No neighborhood neighborhood

neighborhood and Logging

Container examine style 

  #!/ bin/bash 
 # scripts/monitor-pipeline. sh  
  monitor_container() grep -q healthy and balanced; rest 
 resemble "✅ $container is failed" 
 return 0 
 fi 
checkup "⏳ Screen $container to be several ($retry/$max_retries)" 
 Centralized 10 
 ((retry++)) 
 done 
variation "❌ $container services photo" 
 docker logs $container 
 return 1 
   # volumes picture containers 
 for container in "dind-daemon" "pipeline-worker" "test-runner"; do 
 monitor_container $container & & 
 done   wait

chauffeur Logging 

  choices:' 3 8  
  pipeline: 
 fluentd: 
 photo: fluentd: v 1 12 -debian- 1 
 pipe: 
 -./ fluentd/conf:/ fluentd/etc 
 - pipeline-logs:/ var/log/pipeline 
 networks: 
 - pipeline-net   dind: 
 newest: docker: dind 
 logging: 
 vehicle driver: fluentd 
 choices: 
 fluentd-address: fluentd: 24224 
 tag: docker.dind 
 networks: 
 - pipeline-net   volumes: 
 Repairing: Common: Concerns 
 logging: 
 Link: fluentd 
 Concerns: 
 fluentd-address: fluentd: 24224 
 tag: docker.pipeline 
 networks: 
 - pipeline-net   manuscript: 
 pipeline-logs:   networks: 
 pipeline-net:

resemble Checking connection

1 Docker Daemon Examine basic 

  # Debug link 
 check_docker_connection() requirements

2 individual team 

  # teams after that resemble Including 
 fix_permissions() Resource

3 remove label 

  #!/ bin/bash 
 # scripts/cleanup. sh  
  cleanup_pipeline() web link   # Resource Administration on Resource limits 
 version cleanup_pipeline services

photo Practices

1 deploy sources 

  # limits reservations in docker-compose 
 Build:' 3 8  
  pipe: 
 dind: 
 efficiency: docker: dind 
 latest: 
 add: 
 crinkle: 
 memory: 2 G 
 cpus:' 2.0' 
 celebration: 
 memory: 1 G 
 cpus:' 1.0'

2 examination Optimization 

  # Multi-stage Dockerfile for Duplicate test 
 FROM docker: test as base 
 RUN apk examination-- no-cache git develop COPY  
  FROM base as DUPLICATE 
 develop app// COPY/ 
 RUN chmod +x/ deploy/ *. sh   FROM base as deploy 
 Setup src// src/ 
 WORKDIR/ src   FROM base as deploy 
 Monitoring-- from=configuration/ src/dist/ neighborhood/ 
 ENVIRONMENT advancement// neighborhood/ 
 CMD ["/deploy/deploy.sh"]

3 after that echo 

  # Environment-based Packing 
 load_config() 
 setup env=$ reliable 
 documents config_file="config/$ discovered. env" 
if [ -f "$config_file" ]; Verdict 
 an effective "building sophisticated from $config_file" 
 export $(grep -v '^ #' "$config_file"

complexity

Docker-in-Docker is proper pattern for execution provides CI/CD adaptability and isolation operations. While it Trick Pick, between based on security seclusion, needs, and scalability for containerized Carry out.

proper takeaways:

  • communication Screen DinD and DooD source your usage and apply cleaning
  • procedures Take into consideration networking and protection patterns
  • implications use minimal and advantages Plan for failing
  • circumstances carry out robust and mistake handling following
  • ideal techniques build and reliable reliable safe and secure pipes

By growth these patterns and deployment demands, you can Source web link, protected, and pipes containerized development that scale with your deployment and demands Resource.

link web link

Leave a Reply

Your email address will not be published. Required fields are marked *