Build configuration file schema
A build config file contains instructions for Cloud Build to perform tasks based on your specifications. For example, your build config file can contain instructions to build, package, and push Docker images.
This page explains the schema of the Cloud Build configuration file. For instructions on creating and using a build config file, see Creating a basic build config file.
Structure of a build config file
Build config files are modeled using the Cloud Build API's Build resource.
You can write the build config file using the YAML or the JSON syntax. If you submit build requests using third-party http tools such as curl, use the JSON syntax.
A build config file has the following structure:
Each of the sections of the build config file defines a part of the task you want Cloud Build to execute:
Build steps
A build step specifies an action that you want Cloud Build to perform. For each build step, Cloud Build executes a docker container as an instance of docker run. Build steps are analogous to commands in a script and provide you with the flexibility of executing arbitrary instructions in your build. If you can package a build tool into a container, Cloud Build can execute it as part of your build. By default, Cloud Build executes all steps of a build serially on the same machine. If you have steps that can run concurrently, use the waitFor option.
You can include up to 300 build steps in your config file.
Use the steps field in the build config file to specify a build step. Here's a snippet of the kind of configuration you might set in the steps field:
name
Use the name field of a build step to specify a cloud builder, which is a container image running common tools. You use a builder in a build step to execute your tasks.
The following snippet shows build steps calling the bazel, gcloud, and docker builders:
args
The args field of a build step takes a list of arguments and passes them to the builder referenced by the name field. Arguments passed to the builder are passed to the tool that's running in the builder, which allows you to invoke any command supported by the tool. If the builder used in the build step has an entrypoint, args will be used as arguments to that entrypoint. If the builder does not define an entrypoint, the first element in args will be used as the entrypoint, and the remainder will be used as arguments.
You can create up to 100 arguments per step. The maximum argument length is 10,000 characters.
The following snippet invokes the docker build command and installs Maven dependencies:
env
The env field of a build step takes a list of environment variables to be used when running the step. The variables are of the form KEY=VALUE.
In the following build config the env field of the build step sets the Compute Engine zone and the GKE cluster prior to executing kubectl:
dir
Use the dir field in a build step to set a working directory to use when running the step's container. If you set the dir field in the build step, the working directory is set to /workspace/<dir>. If this value is a relative path, it is relative to the build's working directory. If this value is absolute, it may be outside the build's working directory, in which case the contents of the path may not be persisted across build step executions (unless a volume for that path is specified).
The following code snippet sets the working directory for the build step as /workspace/examples/hello_world:
timeout
Use the timeout field in a build step to set a time limit for executing the step. If you don't set this field, the step has no time limit and will be allowed to run until either it completes or the build itself times out. The timeout field in a build step must not exceed the timeout value specified for a build. timeout must be specified in seconds with up to nine fractional digits, terminated by 's'. Example: 3.5s
In the following build config, the ubuntu step is timed out after 500 seconds:
script
Use the script field in a build step to specify a shell script to execute in the step. If you specify script in a build step, you cannot specify args or entrypoint in the same step. For instructions on using the script field, see Running bash scripts.
id
Use the id field to set a unique identifier for a build step. id is used with the waitFor field to configure the order in which build steps should be run. For instructions on using waitFor and id, see Configuring build step order.
waitFor
Use the waitFor field in a build step to specify which steps must run before the build step is run. If no values are provided for waitFor, the build step waits for all prior build steps in the build request to complete successfully before running. For instructions on using waitFor and id, see Configuring build step order.
entrypoint
Use the entrypoint in a build step to specify an entrypoint if you don't want to use the default entrypoint of the builder. If you don't set this field, Cloud Build will use the builder's entrypoint. The following snippet sets the entrypoints for the npm build step:
secretEnv
A list of environment variables which are encrypted using a Cloud KMS crypto key. These values must be specified in the build's secrets. For information on using this field see Using the encrypted variable in build requests.
volumes
A Volume is a Docker container volume that is mounted into build steps to persist files across build steps. When Cloud Build runs a build step, it automatically mounts a workspace volume into /workspace. You can specify additional volumes to be mounted into your build steps' containers using the volumes field for your steps.
For example, the following build config file writes a file into a volume in the first step and reads it in the second step. If these steps did not specify /persistent_volume path as a persistent volume, the first step would write the file at that path, then that file would be discarded before the second step executes. By specifying the volume with the same name in both steps, the contents of /persistent_volume in the first step are persisted to the second step.
allowFailure
In a build step, if you set the value of the allowFailure field to true, and the build step fails, then the build succeeds as long as all other build steps in that build succeed.
If all of the build steps in a build have allowFailure set to true and all of the build steps fail, then the status of the build is still Successful.
allowExitCodes takes precedence over this field.
The following code snippet allows the build to succeed when the first step fails:
allowExitCodes
Use the allowExitCodes field to specify that a build step failure can be ignored when that step returns a particular exit code.
If a build step fails with an exit code matching the value that you have provided in allowExitCodes, Cloud Build will allow this build step to fail without failing your entire build.
If 100% of your build steps fail, but every step exits with a code that you have specified in the allowExitCodes field, then the build is still successful.
However, if the build step fails, and it produces another exit code -- one that does not match the value you have specified is in allowExitCodes -- then the overall build will fail.
The exit code(s) relevant to your build depend on your software. For example, "1" is a common exit code in Linux. You can also define your own exit codes in your scripts. The allowExitCodes field accepts numbers up to a maximum of 255.
This field takes precedence over allowFailure.
The following code snippet allows the build to succeed when the first step fails with one of the provided exit codes:
timeout
Use the timeout field for a build to specify the amount of time that the build must be allowed to run, to second granularity. If this time elapses, work on the build will cease and the build status will be TIMEOUT. If timeout is not set, a default timeout of 60 minutes will apply to the build. The maximum value that can be applied to timeout is 24 hours. timeout must be specified in seconds with up to nine fractional digits, terminated by 's'. Example: 3.5s
In the following snippet, timeout is set to 660 seconds to avoid the build from timing out because of the sleep:
queueTtl
Use the queueTtl field to specify the amount of time a build can be queued. If a build is in the queue for longer than the value set in queueTtl, the build expires and the build status is set to EXPIRED. If no value is provided, Cloud Build uses the default value of 3600s (1 hour). queueTtl starts ticking from createTime. queueTtl must be specified in seconds with up to nine fractional digits, terminated by 's', for example, 3.5s.
In the following snippet timeout is set to 20s and queueTtl is set to 10s. queueTtl starts ticking at createTime, which is the time the build is requested, and timeout starts ticking at startTime, which is the time the build starts. Therefore, queueTtl will expire at createTime + 10s unless the build starts by then.
logsBucket
Set the logsBucket field for a build to specify a Cloud Storage bucket where logs must be written. If you don't set this field, Cloud Build will use a default bucket to store your build logs.
The following snippet sets a logs bucket to store the build logs:
options
Use the options field to specify the following optional arguments for your build:
env: A list of global environment variable definitions that will exist for all build steps in this build. If a variable is defined in both globally and in a build step, the variable will use the build step value. The elements are of the form KEY=VALUE for the environment variable KEY being given the value VALUE.
secretEnv: A list of global environment variables, encrypted using a Cloud Key Management Service crypto key, that will be available to all build steps in this build. These values must be specified in the build's Secret.
volumes: A list of volumes to mount globally for ALL build steps. Each volume is created as an empty volume prior to starting the build process. Upon completion of the build, volumes and their contents are discarded. Global volume names and paths cannot conflict with the volumes defined a build step. Using a global volume in a build with only one step is not valid as it signifies a build request with an incorrect configuration.
sourceProvenanceHash: Set the sourceProvenanceHash option to specify the hash algorithm for source provenance. The following snippet specifies that the hash algorithm is SHA256:
machineType: Cloud Build provides four high-CPU virtual machine types to run your builds: two machine types with 8 CPUs and two machine types with 32 CPUs. Cloud Build also provides two additional virtual machine types with 1 CPU and 2 CPUs to run your builds. The default machine type is e2-standard-2 with 2 CPUs. Requesting a high-CPU virtual machine may increase the startup time of your build. Add the machineType option to request a virtual machine with a higher CPU:
For more information on using the machineType option see Speeding up builds.
diskSizeGb: Use the diskSizeGb option to request a custom disk size for your build. The maximum size you can request is 2000 GB.
The following snippet requests a disk size of 200 GB:
logStreamingOption: Use this option to specify if you want to stream build logs to Cloud Storage. By default, Cloud Build collects build logs on build completion; this option specifies if you want to stream build logs in real time through the build process. The following snippet specifies that build logs are streamed to Cloud Storage:
logging: Use this option to specify if you want to store logs in Cloud Logging or Cloud Storage. If you do not set this option, Cloud Build stores the logs in both Cloud Logging and Cloud Storage. You can set the logging option to GCS_ONLY to store the logs only in Cloud Storage. The following snippet specifies that the logs are stored in Cloud Storage:
defaultLogsBucketBehavior: The defaultLogsBucketBehavior option lets you configure Cloud Build to create a default logs bucket within your own project in the same region as your build. For more information, see Store build logs in a user-owned and regionalized bucket.
The following build config sets the defaultLogsBucketBehavior field to the value REGIONAL_USER_OWNED_BUCKET:
dynamic_substitutions: Use this option to explicitly enable or disable bash parameter expansion in substitutions. If your build is invoked by a trigger, the dynamic_substitutions field is always set to true and does not need to be specified in your build config file. If your build is invoked manually, you must set the dynamic_substitutions field to true for bash parameter expansions to be interpreted when running your build.
substitutionOption: You'll set this option along with the substitutions field below to specify the behavior when there is an error in the substitution checks.
pool: Set the value of this field to the resource name of the private pool to run the build. For instructions on running a build on a private pool, see Running builds in a private pool.
requestedVerifyOption: Set the value of requestedVerifyOption to VERIFIED to verify the generation of attestations and provenance metadata for your build. This option also enables attestations and provenance metadata for regional builds and builds in private pools. If you don't add requestedVerifyOption: VERIFIED, Cloud Build generates provenance for global builds only.
substitutions
Use substitutions in your build config file to substitute specific variables at build time. Substitutions are helpful for variables whose value isn't known until build time, or to re-use an existing build request with different variable values. By default, the build returns an error if there's a missing substitution variable or a missing substitution. However, you can use the ALLOW_LOOSE option to skip this check.
The following snippet uses substitutions to print "hello world." The ALLOW_LOOSE substitution option is set, which means the build will not return an error if there's a missing substitution variable or a missing substitution.
For additional instructions on using substitutions, see Substituting variable values.
tags
Use the tags field to organize your builds into groups and to filter your builds. The following config sets two tags named mytag1 and mytag2:
availableSecrets
Use this field to use a secret from Secret Manager with Cloud Build. For more information, see Using secrets.
secrets
Secret pairs a set of secret environment variables containing encrypted values with the Cloud KMS key to use to decrypt the value.
serviceAccount
Use this field to specify the IAM service account to use at build time. For more information, see Configuring user-specified service accounts.
images
The images field in the build config file specifies one or more Linux Docker images to be pushed by Cloud Build to Artifact Registry or Container Registry (Deprecated). You may have a build that performs tasks without producing any Linux Docker images, but if you build images and don't push them to the registry, the images are discarded on build completion. If a specified image is not produced during the build, the build will fail. For more information on storing images, see Store artifacts in Artifact Registry.
The following build config sets the images field to store the built image:
artifacts
The artifacts field in the build config file specifies one or more non-container artifacts to be stored in Cloud Storage. For more information on storing non-container artifacts, see Store build artifacts in Cloud Storage.
The following build config sets the artifacts field to store the built Go package to gs://mybucket/:
mavenArtifacts
The mavenArtifacts field allows you to upload non-container Java artifacts to Maven repositories in Artifact Registry. For more information, see Build and test Java applications.
The following build config sets the mavenArtifacts field to upload the packaged file my-app-1.0-SNAPSHOT.jar to the Artifact Registry repository https://us-central1-maven.pkg.dev/my-project-id/my-java-repo:
pythonPackages
The pythonPackages field allows you to upload Python packages to Artifact Registry. For more information, see Build and test Python applications.
The following build config sets the pythonPackages field to upload the Python package dist/my-pkg.whl to the Artifact Registry repository https://us-east1-python.pkg.dev/my-project/my-repo:
npmPackages
Use the npmPackages field to configure Cloud Build to upload your built npm packages to supported repositories in Artifact Registry. You must provide values for repository and packagePath.
The repository field specifies the Artifact Registry repository to store your packages. The packagePath field specifies the local directory that contains the npm package to upload. This directory must contain a package.json file.
We recommend using an absolute path for the value of packagePath. You can use . to refer to the current working directory, but the field cannot be omitted or left empty. For more instructions on using npmPackages, see Build and test Node.js applications.
The following build config sets the npmPackages field to upload the npm package in the /workspace/my-pkg directory to the Artifact Registry repository https://us-east1-npm.pkg.dev/my-project/my-repo.
Using Dockerfiles
If you're executing Docker builds in Cloud Build using the gcloud CLI or build triggers, you can use a Dockerfile without a separate build config file. If you wish to make more adjustments to your Docker builds, you can provide a build config file in addition to the Dockerfile. For instructions on how to build a Docker image using a Dockerfile, see Quickstart: Build.
Cloud Build network
When Cloud Build runs each build step, it attaches the step's container to a local Docker network named cloudbuild. The cloudbuild network hosts Application Default Credentials (ADC) that Google Cloud services can use to automatically find your credentials. If you're running nested Docker containers and want to expose ADC to an underlying container or using gsutilor gcloud in a docker step, use the --network flag in your docker build step:
Comments
Post a Comment