• setup node, java, python

name: setup-time-calcul

on:
  workflow_dispatch:
    inputs:
      BUILD_MACHINE:
        description: '빌드를 진행할 머신을 선택해주세요(택1)'
        type: choice
        default: k8s-runner
        options:
          - k8s-runner
          - ubuntu-latest
      DEBUG:
        description: 'debug 세션 활성화'
        type: boolean
        required: false

jobs:
  build_runner_image:
    runs-on: ${{ inputs.BUILD_MACHINE }}
    steps:
      - name: checkout src
        uses: actions/checkout@v4

      - name: Login to DockerHub
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}

      - name: Setup python 3.13
        uses: actions/setup-python@v4
        with:
          python-version: '3.13' 
      - name: Setup Node 18
        uses: actions/setup-node@v4
        with:
          node-version: 18.x
      - name: Setup Java
        uses: actions/setup-java@v4
        with:
          distribution: 'temurin'
          java-version: '11'

      - name: make debug-session
        if: ${{ failure() || inputs.DEBUG == 'true' }}
        run: |
          screen -dmS debug bash
          echo "Debug session started. To end normally, create /tmp/end_debug file."
          echo "Runner name: ${{ runner.name }}"
          echo "To connect: screen -r debug"
          echo "To end : touch /tmp/end_debug"
          while [ ! -f /tmp/end_debug ]; do sleep 10; done
          echo "Debug session ended normally."

• 기존에는 사용도구마다 일정의 소요시간 발생
  • 사용환경에 따라 금방 설치되기도하고 다소 시간이 걸리기도함
  • 또한 반복적인 환경구성의 경우, dockerhub pull limit에 걸리기도함(mysql, mongodb, etc..)

• Runner Shell 내부로 접근시 캐싱된 도구가 actions-tool-cache 디렉토리에 저장되어있음을 확인 가능

환경 구성에 사용된 tool을 캐싱하여 GItHub 아티팩트에 저장하고, Runner 이미지 빌드시 해당 아티팩트를 가져와 재사용하는 구성

• Dockerfile

FROM ghcr.io/actions/actions-runner:latest
ENV ACTIONS_TOOL_CACHE=/home/runner/actions-tool-cache
COPY --link --chown=1001:123 tools $ACTIONS_TOOL_CACHE

• Building a tool cache

on:
 # Your triggers here

jobs:
  create-tool-cache:
    runs-on: ubuntu-latest
    steps:

      ## Remove any existing cached content
      - name: Clear any existing tool cache
        run: |
          mv "${{ runner.tool_cache }}" "${{ runner.tool_cache }}.old"
          mkdir -p "${{ runner.tool_cache }}"          
      
      ## Run the setup tasks to download and cache the required tools
      - name: Setup Node 16
        uses: actions/setup-node@v4
        with:
          node-version: 16.x
      - name: Setup Node 18
        uses: actions/setup-node@v4
        with:
          node-version: 18.x
      - name: Setup Java
        uses: actions/setup-java@v4
        with:
          distribution: 'temurin'
          java-version: '21'

      ## Compress the tool cache folder for faster upload
      - name: Archive tool cache
        working-directory: ${{ runner.tool_cache }}
        run: |
          tar -czf tool_cache.tar.gz *          

      ## Upload the archive as an artifact
      - name: Upload tool cache artifact
        uses: actions/upload-artifact@v4
        with:
          name: tools
          retention-days: 1
          path: ${{runner.tool_cache}}/tool_cache.tar.gz

• Build the image

build-with-tool-cache:
    runs-on: ubuntu-latest

    ## We need the tools archive to have been created
    needs: create-tool-cache
    env:
      # Setup some variables for naming the image automatically
      REGISTRY: ghcr.io
      IMAGE_NAME: ${{ github.repository }}

    steps:
    
      ## Checkout the repo to get the Dockerfile 
      - name: Checkout repository
        uses: actions/checkout@v4

      ## Download the tools artifact created in the last job
      - name: Download artifacts
        uses: actions/download-artifact@v4
        with:
          name: tools
          path: ${{github.workspace}}/tools

      ## Expand the tools into the expected folder
      - name: Unpack tools
        run: |
          tar -xzf ${{github.workspace}}/tools/tool_cache.tar.gz -C ${{github.workspace}}/tools/
          rm ${{github.workspace}}/tools/tool_cache.tar.gz          

      ## Build the image

      ## Set up BuildKit Docker container builder
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
      
      ## Automatically create metadata for the image
      - name: Extract Docker metadata
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}

      ## Build the image
      - name: Build and push Docker image
        id: build
        uses: docker/build-push-action@v5
        with:
          context: .
          push: false
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}

• 이미 Runner Docker Image 내부에 지정 버전의 도구가 설정되어 체크 후 패스

• hostmachine과 /data 디렉토리를 공유하는 pv-runner 러너그룹 생성

# helm upgrade pv-runner oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set -f ./pv-runner.yaml --version 0.9.3 -n arc-systems -i

# runner group을 연결할 github repository/organization 등록
githubConfigUrl: https://github.com/nasir17git/logonme
githubConfigSecret: arc-ghp-token
  #github_token: "ghp_aaa" //직접 토큰값을 넣을경우

# 유지할 Runner 수 설정
maxRunners: 3
minRunners: 3

controllerServiceAccount:
  namespace: arc-system
  name: arc-controller-gha-rs-controller

containerMode:
  type: "dind" 

template:
  spec:
    containers:
      - name: runner
        image: nasir17/gha-demo:latest
        command: ["/home/runner/run.sh"]
    # hostmachine의 directory
    volumes:
    - name: runner-pv
      hostPath:
        type: DirectoryOrCreate
        path: /data/runner
    - name: docker-pv
      hostPath:
        type: DirectoryOrCreate
        path: /data/docker
    topologySpreadConstraints:
    - labelSelector:
        matchLabels:
          actions.github.com/scale-set-name: pv-runner
      maxSkew: 1
      topologyKey: kubernetes.io/hostname
      whenUnsatisfiable: ScheduleAnyway
    tolerations:
      - key: node-role.kubernetes.io/control-plane
        operator: Exists
        effect: NoSchedule

• 생성될 볼륨을 부착하는 volumeMount는 values에서 제어불가
• 수동으로 pv-runner의 autoscalingrunnerset 에서 수정

...
        image: nasir17/gha-demo:latest
        name: runner
        volumeMounts:
        - mountPath: /home/runner/_work
          name: work
        - mountPath: /var/run
          name: dind-sock
        - mountPath: /home/runner/actions-tool-cache
          name: runner-pv
...
        image: docker:dind
        name: dind
        securityContext:
          privileged: true
        volumeMounts:
        - mountPath: /home/runner/_work
          name: work
        - mountPath: /var/run
          name: dind-sock
        - mountPath: /home/runner/externals
          name: dind-externals
        - name: docker-pv
          mountPath: /var/lib/docker
...

• runner-pv > 호스트의 data파일만 남아있음
  • 사전에 runner의 worker 위치 파악후 docker exec -it nasir-k8s-worker2 touch /data/runner/test.txt
• docker image에 저장된 tool 이 아닌 node의 /data/runner 와 연결
• docker image들 또한 저장되어있지 않음

• 테스트를 위해 러너 내부에서 docker image 저장 후, runner 환경구성(setup-time-calcul) 워크플로우 실행
  • docker pull nginx
• 이후 확인
  • 이전 작동 이후, 같은 노드에 재생성된 runner의 경우 이전에 구성했던 docker image와 tool이 해당 디렉토리에 남아있음