Interacting with Docker Hub images

 You can use container images from Docker Hub to execute your tasks in Cloud Build. Additionally, if your build produces images, you can push them to Docker Hub. This page describes how to write build config files to push and pull Docker Hub images. For an overview of all the fields available in a build config file, see Build Configuration Overview.

You can pull official Docker images, Docker-certified images, and custom images stored in Docker Hub in your build step by specifying the name of the image in the name field. Cloud Build will first pull the specified image from Docker Hub and then use the image to run the build step.

In the example below, Cloud Build pulls the Docker image for maven to run the mvn command specified in args:

YAMLJSON
steps:
- name: "maven"
  args: ["mvn", "--version"]

To pull private images and to push public and private images to Docker Hub, Cloud Build will need to authenticate to Docker with your credentials. To include Docker credentials in your builds, you must first store your credentials in Secret Manager, and then grant permission for Cloud Build to access the secret from Secret Manager.

To store Docker credentials in Secret Manager:

  1. Go to the Secret Manager page in the Google Cloud console:

    Go to the Secret Manager page

  2. On the Secret Manager page, click Create Secret.

  3. On the Create secret page, under Name, enter docker-username.

  4. In the Secret value field, enter your Docker username.

  5. Leave the Regions section unchanged.

  6. Click the Create secret button.

Repeat the steps above to store your Docker password in Secret Manager.

To grant the Secret Manager Secret Accessor IAM role for the secret to the service account you are using for the build:

  1. Open the Secret Manager page in the Google Cloud console:

    Go to the Secret Manager page

  2. Select the checkbox of the secret corresponding to your Docker username and password.

  3. If it is not already open, click Show info panel to open the panel.

  4. In the panel, under Permissions, click Add principal.

  5. In the New principals field, enter the email address of your service account.

  6. In the Select a role drop-down box, select Secret Manager Secret Accessor role.

  7. Click Save.

To pull private images from Docker Hub:

  1. Make sure you've stored your Docker credentials in Secret Manager and granted permissions for Cloud Build to access the secret.

  2. In the build config file:

    • After all the build steps, add an availableSecrets field specifying the secret version and the env variable for the Docker username and password.
    • In the build step where you want to specify the username and password:
      • Add an entrypoint field pointing to bash to use the bash tool in the build step. This is required to refer to the environment variable for the secret.
      • Add a secretEnv field specifying the environment variable for username and password.
      • In the args field, add a -c flag as the first argument. Any string you pass after -c is treated as a command. For more information on running bash commands with -c, see the bash documentation.
      • When specifying the secret in the args field, specify it using the environment variable prefixed with $$.

    The following build config file shows how to login to Docker using the Docker username and password stored in Secret Manager, and run a private image.

    YAMLJSON
     steps:
         - name: 'gcr.io/cloud-builders/docker'
       entrypoint    : 'bash'
       args    : ['-c', 'docker login --username=$$USERNAME --password=$$PASSWORD']
       secretEnv    : ['USERNAME', 'PASSWORD']
         - name: "gcr.io/cloud-builders/docker"
       entrypoint    : 'bash'
       args    : ['-c', 'docker run $$USERNAME/REPOSITORY:TAG']
       secretEnv    : ['USERNAME']
     availableSecrets    :
       secretManager    :
           - versionName: projects/PROJECT_ID/secrets/DOCKER_PASSWORD_SECRET_NAME/versions/DOCKER_PASSWORD_SECRET_VERSION
         env    : 'PASSWORD'
           - versionName: projects/PROJECT_ID/secrets/DOCKER_USERNAME_SECRET_NAME/versions/DOCKER_USERNAME_SECRET_VERSION
         env    : 'USERNAME'

    Replace the placeholder values in the above commands with the following:

    • PROJECT_ID: The ID of the Google Cloud project where you've stored your secrets.
    • DOCKER_USERNAME_SECRET_NAME: The secret name corresponding to your Docker username.
    • DOCKER_USERNAME_SECRET_VERSION: The secret version of your Docker username.
    • DOCKER_PASSWORD_SECRET_NAME: The secret name corresponding to your Docker password.
    • DOCKER_PASSWORD_SECRET_VERSION: The secret version of your Docker password.
    • REPOSITORY: The name of your Docker repository from where you're pulling the image.
    • TAG: The tag name of your image.

  3. Use the build config file to manually start a build or to automate builds using triggers.

To push public and private images to Docker Hub:

  1. Make sure you've stored your Docker credentials in Secret Manager and granted permissions for Cloud Build to access the secret.

  2. In the build config file:

    • After all the build steps, add an availableSecrets field specifying the secret version and the env variable for the Docker username and password.
    • In the build step where you want to specify the username and password:
      • Add an entrypoint field pointing to bash to use the bash tool in the build step. This is required to refer to the environment variable for the secret.
      • Add a secretEnv field specifying the environment variable for username and password.
      • In the args field, add a -c flag as the first argument. Any string you pass after -c is treated as a command. For more information on running bash commands with -c, see the bash documentation.
      • When specifying the secret in the args field, specify it using the environment variable prefixed with $$.

    The following example build config file shows how to login to Docker, build an image with source code stored locally, and then push the image to Docker repository.

    YAMLJSON
     steps:
         - name: 'gcr.io/cloud-builders/docker'
       entrypoint    : 'bash'
       args    : ['-c', 'docker login --username=$$USERNAME --password=$$PASSWORD']
       secretEnv    : ['USERNAME', 'PASSWORD']
         - name: 'gcr.io/cloud-builders/docker'
       entrypoint    : 'bash'
       args    : ['-c', 'docker build -t $$USERNAME/REPOSITORY:TAG .']
       secretEnv    : ['USERNAME']
         - name: 'gcr.io/cloud-builders/docker'
       entrypoint    : 'bash'
       args    : ['-c', 'docker push $$USERNAME/REPOSITORY:TAG']
       secretEnv    : ['USERNAME']
     availableSecrets    :
       secretManager    :
           - versionName: projects/PROJECT_ID/secrets/DOCKER_PASSWORD_SECRET_NAME/versions/DOCKER_PASSWORD_SECRET_VERSION
         env    : 'PASSWORD'
           - versionName: projects/PROJECT_ID/secrets/DOCKER_USERNAME_SECRET_NAME/versions/DOCKER_USERNAME_SECRET_VERSION
         env    : 'USERNAME'

    Replace the placeholder values in the above commands with the following:

    • PROJECT_ID: The ID of the Google Cloud project where you've stored your secrets.
    • DOCKER_USERNAME_SECRET_NAME: The secret name corresponding to your Docker username.
    • DOCKER_USERNAME_SECRET_VERSION: The secret version of your Docker username.
    • DOCKER_PASSWORD_SECRET_NAME: The secret name corresponding to your Docker password.
    • DOCKER_PASSWORD_SECRET_VERSION: The secret version of your Docker password.
    • REPOSITORY: The name of your Docker repository to which you're pushing the image.
    • TAG: The tag name of your image.

  3. Use the build config file to manually start a build or to automate builds using triggers.

The supported Docker builder for Cloud Build, gcr.io/cloud-builders/docker uses Docker 20.10.14. With this version, if you don't specify a tag when pushing an image to Docker, Docker pushes only the image with the latest tag. If the latest tag doesn't exist, the push fails.

To push an image with a specific tag to Docker, specify the tag in the docker push build step. The following example pushes an image tagged prod:

YAMLJSON
steps:
...
- name: 'gcr.io/cloud-builders/docker'
  args: ['docker', 'push', '$$USERNAME/myrepo:prod']
...

To push all tags of an image to Docker, add the -a flag to the list of args in the docker push build step:

YAMLJSON
steps:
...
- name: 'gcr.io/cloud-builders/docker'
  args: ['docker', 'push', '-a', '$$USERNAME/myrepo']
...

You can use Docker client 19.03.9 by tagging the version in the Docker builder:

YAMLJSON
steps:
...
- name: 'gcr.io/cloud-builders/docker:19.03.9'
  args: ['docker', 'push', '$$USERNAME/myrepo:prod']
...

Comments

Post a Comment

Popular posts from this blog

Terraform

Image
  Terraform is an infrastructure as code tool that lets you build, change, and version cloud and on-prem resources safely and efficiently. HashiCorp Terraform is an infrastructure as code tool that lets you define both cloud and on-prem resources in human-readable configuration files that you can version, reuse, and share. You can then use a consistent workflow to provision and manage all of your infrastructure throughout its lifecycle. Terraform can manage low-level components like compute, storage, and networking resources, as well as high-level components like DNS entries and SaaS features. How does Terraform work? Terraform creates and manages resources on cloud platforms and other services through their application programming interfaces (APIs). Providers enable Terraform to work with virtually any platform or service with an accessible API. HashiCorp and the Terraform community have already written  thousands of providers  to manage many different types of resources...
Read more

Different Types of Reports in Scrum - Agile

Image
  Introduction to Agile Reporting I’m not going to elaborate on the Agile Framework itself here.  1. Sprint Burndown At a Sprint-level, the burndown presents the  easiest way to track and report status  (the proverbial  Red / Amber / Green ), i.e., whether your Sprint is on or off-track, and what are the chances of meeting the Sprint goals. The burndown chart – when used right – can provide  near-real time updates  on Sprint progress. “If your team do it right, then they would take in just the right amount of work into a sprint.” At the beginning of a Sprint, the Scrum team perform Sprint Planning and agree to take on development work worth a certain number of Story points. This forms the basis for the Sprint Burndown chart. The total story points agreed at the beginning of the sprint make up the y-axis, and the individual dates in the Sprint make up x-axis. If your team do it right, then they would take in just the right amount of work into ...
Read more

Scrum Master Interview help - Bootcamp

Understanding the MoSCoW prioritization | How to implement it into your project Projects, irrespective of their size or complexity, often juggle numerous tasks and requirements. In such a scenario,  MoSCoW prioritization  becomes essential to ensure the successful completion of the project.  Besides, it is also a very effective way to manage project priorities. With a straightforward and adaptable approach, the MoSCoW method helps manage stakeholder expectations and improve project outcomes. What is the MoSCoW prioritization technique?  The MoSCoW method is a prioritization matrix widely used in project management and software development. The term MoSCoW is an acronym that stands for  Must have ,  Should have ,  Could have , and  Won’t have , each denoting a level of priority. Here is the breakdown of the MoSCoW method:  Must have:  These are critical requirements that the project cannot be completed withou...
Read more