Setup Private Pool - Google Build
Overview of default pools and private pools
By default, when you run a build on Cloud Build, the build runs in a secure, hosted environment with access to the public internet. Each build runs on its own worker and is isolated from other workloads. You can customize your build in multiple ways including increasing the size of the machine type or allocating more disk space. The default pool has limits on how much you can customize the environment, particularly around private network access.
Private pools are private, dedicated pools of workers that offer greater customization over the build environment, including the ability to access resources in a private network. Private pools, similar to default pools, are hosted and fully-managed by Cloud Build and scale up and down to zero, with no infrastructure to set up, upgrade, or scale. Because private pools are customer-specific resources, you can configure them in more ways.
The following tables compares the features of the default pool with a private pool:
| Feature | Default pool | Private pool |
|---|---|---|
| Fully managed | ||
| Pay per build minute | ||
| Auto scaling, scales to 0 | ||
| Access to public internet | Configurable | |
| Peer into a VPC or shared VPC to access private resources | ||
| VPC Service Controls support | ||
| Static internal IP ranges | ||
| Ability to disable public IPs | ||
| Maximum concurrent builds | 30 | 100+ |
| Machine types | 5 | 15 |
| Regions | global, supported regions | supported regions |
| Region in which the build is run | The build runs in the region specified in gcloud builds submit or in the Google Cloud console triggers page. | The build runs in the region where you create the private pool. |
Network architecture
Private pool is a fully-managed resource created by a user in a Google Cloud project. Private pools can be created, updated, and deleted via API, Google Cloud CLI, Google Cloud console, or managed via infrastructure provisioning tools such as Terraform.
Using Identity and Access Management permissions, builds run on private pools can be submitted cross project; that is builds don't have to originate from the same project as the private pool. Likewise, the service account the build runs as can be configured to allow workers to access resources from other Google Cloud projects during a build.
Private pools are hosted in a Google-owned service producer network that comes with internal IP addresses. For your builds to access resources in your private network, you can create a private VPC peering connection between your VPC network and the service producer network.
The following diagram shows how builds are routed to the private pool. In this setup, the customer has two projects sending builds to Cloud Build. The customer has configured all the builds from these projects to be routed to their private pool. The service producer network, where the private pool is in, is peered to the customer's VPC network giving the builds access to private resources in the customer's VPC network.
Create and manage private pools :
Before you begin
Create a new Google Cloud project or choose an existing project. You'll use this project to create the private pool.
Enable the Cloud Build API.
To use the command-line examples in this guide, install and configure the Google Cloud CLI.
[Optional] For builds to access private resources from your Virtual Private Cloud network, you must set up a peering connection between your Virtual Private Cloud network and the Virtual Private Cloud network where private pools reside. For instructions, see set up your environment to create private pools.
Create a new Google Cloud project or choose an existing project. You'll use this project to create the private pool.
Enable the Cloud Build API.
To use the command-line examples in this guide, install and configure the Google Cloud CLI.
[Optional] For builds to access private resources from your Virtual Private Cloud network, you must set up a peering connection between your Virtual Private Cloud network and the Virtual Private Cloud network where private pools reside. For instructions, see set up your environment to create private pools.
Creating a new private pool
IAM permissions: You require Cloud Build WorkerPool Owner role to perform this task. For instructions on granting this role, see Configure access to Cloud Build resources.
You can create up to 10 private pools per Google Cloud project per region. To create a new private pool:
Open the Worker Pool page in the Google Cloud console:
Select the project where you want to create the private pool.
In the Worker pool page, click Create.
You will see the Create private pool side panel.
Enter the following information to create your private pool:
Name: Enter a name for your private pool. This value can only contain alphanumeric characters
/[a-z][0-9]/or dashes-. The name of your private pool must be between 1 and 63 characters.Region: Select the region in which you want the private pool to be created.
Machine type: Select the Compute Engine machine type you want to use for your private pool.
Available disk size: Enter a disk size for your private pool. Specify a value greater than or equal to 100 and less than or equal to 1000. If not provided, Cloud Build uses a disk size of 100.
Under Network type, select one of the following options:
Default network: Select this option if your instance is accessible via the public internet. When the Default network option is selected, your private pool uses the service producer network. For more information, see Set up environment to use private pools in a VPC network.
Private network: Select this option if your instance is hosted on a private network.
Project: Select your Google Cloud project ID.
Network: Select your network from the drop-down menu. If you have not created a network, see Create and manage VPC networks to learn how to create a network.
IP range: Enter the internal IP range that the Cloud Build producer network can use to allocate to VMs maintaining a connection with private repositories.
You can specify the range using the Classless Inter-Domain Routing (CIDR) routing notation in the format
STARTING_IP_ADDRESS/SUBNET_PREFIX_SIZE. For example,192.0.2.0/24has a prefix length of 24. The first 24 bits of the IP range are used as the subnet mask (192.0.2.0) while the possible hosts addresses range from192.0.2.0to192.0.2.255.The value of your prefix length must not exceed
/29. If no value is specified for the range, a default value of/24is automatically assigned. If no value is specified for the prefix length, IP addresses are automatically assigned within the peered VPC network. If no value is specified for the IP address, the IP address is automatically assigned a range within the peered VPC network.
Assign external IPs: This option is selected by default to allow private pools to access the public internet. Uncheck this box to restrict access to your private network.
Click Create to create your private pool.
Creating a private pool within a VPC Service Controls perimeter
If you're creating a private pool within a VPC Service Controls perimeter, see Using VPC Service Controls.
Updating a private pool
IAM permissions: You require Cloud Build WorkerPool Editor role to perform this task. For instructions on granting this role, see Configuring access to Cloud Build resources.
You can update the disk size and machine type of an existing private pool. To update a private pool:
Open the Worker pool page in the Google Cloud console:
Select the project in which you created the private pool.
Click on the private pool name.
In the Edit private pool side panel, update the machine type and/or the disk size.
Click Save.
Viewing the details of your private pool
IAM permissions: You require Cloud Build WorkerPool Viewer role to perform this task. For instructions on granting this role, see Configuring access to Cloud Build resources.
To view the details of a private pool:
Open the Worker pool page in the Google Cloud console:
Select the project in which you created the private pool
Click on the private pool name.
The Edit private pool side panel is displayed, which has the details of the private pool.
Deleting a private pool
IAM permissions: You require Cloud Build WorkerPool Owner role to perform this task. For instructions on granting this role, see Configuring access to Cloud Build resources.
To delete a private pool:
Open the Worker pool page in the Google Cloud console:
In the row with your private pool, click the trash icon.
Private pool configuration file schema :
A private pool configuration file (private pool config file) contains instructions for Cloud Build to create a private pool based on your specifications. A private pool config file contains information such as the disk size and machine type you want to use for your private pool. The private pool config file is required when you create and manage private pools using the gcloud tool or the Cloud Build API.
Structure of a private pool config file
You can write the private pool config file using the YAML or the JSON syntax. To create and manage private pools using curl, write the private pool config file in the JSON format. To create and manage private pools using the gcloud tool, write the private pool config file in the YAML or the JSON format.
A private pool config file has the following structure:
The fields in the private pool config file are explained below:
networkConfig
This is an optional field. Specify networkConfig only if you're peering your VPC network with the service producer network.
egressOption
This is an optional field. Set the value of this field to NO_PUBLIC_EGRESS if you're creating your private pool within a VPC Service Controls perimeter. If you don't include this field in the private pool config file, public egress is enabled. For information on setting up VPC Service Controls for private pools, see Using VPC Service Controls.
peeredNetwork
This is a required field if your private pool config file contains the networkConfig field. Set the value of this field to the network resource URL of the VPC network that is peered to the service producer network. The network resource URL must be of the format projects/NETWORK_PROJECT_ID/global/networks/NETWORK_NAME, where NETWORK_PROJECT_ID is the project ID of the Google Cloud project that holds your VPC network and NETWORK_NAME is the name of your VPC network. If you don't specify a value, Cloud Build uses the service producer network by default.
peeredNetworkIpRange
This is an optional field. Set the value of this field to specify the internal IP range that VMs can be assigned to within the peered network's allocated range. If the specified IP range cannot be allocated within the range of the private connection, for example due to size limitations or an invalid starting IP, the private pool cannot be created.
peeredNetworkIpRange is specified using Classless Inter-Domain Routing (CIDR) notation in the format 'STARTING_IP/SUBNET_PREFIX_SIZE'. If this field is used, SUBNET_PREFIX_SIZE must be specified. SUBNET_PREFIX_SIZE determines the size of the IP range. It must begin with a slash (/) and have a value less than or equal to 29. The STARTING_IP portion is optional, and specifies the starting address of the range. If the STARTING_IP is not specified, a starting address is automatically assigned from within the private connection range. If no value is specified for peeredNetworkIpRange, a default value of /24 will be used which automatically assigns a starting IP with a range of 256 IPs.
The value of SUBNET_PREFIX_SIZE must not exceed 29. A larger number specifies a larger subnet prefix, which would leave fewer possible IPs and result in a smaller IP range. Since an IP address is 32 bits, a prefix size of /24 allows 256 possible IPs in the range (eight bits) whereas a prefix size of /29 leaves eight possible IPs (three bits). The maximum number of builder VMs is limited to the number of usable IPs in the range (the total possible number of IPs in the range minus two).
See the following for examples of the peeredNetworkIpRange field:
192.168.0.0/24specifies an IP range starting at 192.168.0.0 with a subnet prefix size of 24 (256 IPs, with 254 usable IPs)./29specifies a range with an automatically determined starting IP and a subnet prefix size of 29 (eight IPs, with six usable IPs).
workerConfig
This is a required field. It contains configuration options for your private pool.
diskSizeGb
This is an optional field. Use this field to specify a disk size for your private pool instance in GB. Specify a value greater than or equal to 100 and less than or equal to 2000. If you don't include this field in your private pool config file, or if you specify 0 as the value of this field, Cloud Build uses the default value of 100.
machineType
This is an optional field. Use this field to specify the Compute Engine machine type for your private pool instance. Specify one of the following machine types:
e2-mediume2-standard-2e2-standard-4e2-standard-8e2-standard-16e2-standard-32e2-highmem-2e2-highmem-4e2-highmem-8e2-highmem-16e2-highcpu-2e2-highcpu-4e2-highcpu-8e2-highcpu-16e2-highcpu-32
If you don't specify a value, Cloud Build uses the default value of e2-medium.
Run builds in a private pool :
Before you begin
Make sure you've created a private pool using the steps in Creating and managing private pools.
To use the command-line examples in this guide, install and configure the Google Cloud CLI.
You can either use the same Google Cloud project to create your private pool and your Cloud Build builds or use different projects. If your builds are in a different project from your private pool, set the default project in the gcloud CLI to the project where your builds originate:
Make sure you've created a private pool using the steps in Creating and managing private pools.
To use the command-line examples in this guide, install and configure the Google Cloud CLI.
You can either use the same Google Cloud project to create your private pool and your Cloud Build builds or use different projects. If your builds are in a different project from your private pool, set the default project in the gcloud CLI to the project where your builds originate:
IAM permissions
To run builds via the gcloud CLI or the Cloud Build API, grant the WorkerPool User role in the private pool project to the user or service account that requests the build.
To run automated builds using triggers:
- If the project in which you're starting the build is the same as the project in which your private pool exists, you don't need to grant any permissions.
If the project in which you're starting the build is different from the project in which your private pool exists, grant the WorkerPool User role to the service account you are using with your trigger on the workerpool project where your builds are created:
Open the IAM page in the Google Cloud console.
In the project selector drop-down menu at the top of the page, select the project that contains your private pool.
Click Grant access.
Enter the following principal and role settings:
Add principals: Enter email address of the service account that you are using with your trigger.
Assign roles: Select the Cloud Build WorkerPool User role.
Click Save to save your new IAM permissions.
To run builds via the gcloud CLI or the Cloud Build API, grant the WorkerPool User role in the private pool project to the user or service account that requests the build.
To run automated builds using triggers:
- If the project in which you're starting the build is the same as the project in which your private pool exists, you don't need to grant any permissions.
If the project in which you're starting the build is different from the project in which your private pool exists, grant the WorkerPool User role to the service account you are using with your trigger on the workerpool project where your builds are created:
Open the IAM page in the Google Cloud console.
In the project selector drop-down menu at the top of the page, select the project that contains your private pool.
Click Grant access.
Enter the following principal and role settings:
Add principals: Enter email address of the service account that you are using with your trigger.
Assign roles: Select the Cloud Build WorkerPool User role.
Click Save to save your new IAM permissions.
Running builds
You can submit builds from the same Google Cloud project where you created the private pool or from a different Google Cloud project. You can specify the private pool either in your build config file or direcly in the gcloud command:
Specifying the private pool in the build config file:
In your Cloud Build config file, add a
pooloption and specify the full resource name of the private pool to run the build:Replace the placeholder values in the config file above with the following:
PRIVATEPOOL_PROJECT_ID: the Google Cloud project where your private pool is located.REGION: the region where you created your private pool.PRIVATEPOOL_ID: the unique private pool ID that you specified when creating the private pool.
Use the build config file created above to run your build via
gcloudor API or using triggers. If your instance is hosted on-premises, Cloud Build also provides trigger functionality support for several external source code management systems such as GitHub Enterprise or Bitbucket Server.
Specifying the private pool in the gcloud command:
You can specify the private pool in the gcloud command instead of in the build config file. For example, consider you have the following build config file:
The following command builds using the build config file and specifies the worker pool in the command:
gcloud builds submit --config=CONFIG_FILE
--worker-pool=projects/PRIVATEPOOL_PROJECT_ID /locations/REGION /workerPools/PRIVATEPOOL_ID
Replace the placeholder values in the above commands with the following:
CONFIG_FILE: path to your build config file.PRIVATEPOOL_ID: the unique private pool ID that you specified when creating the private pool.PRIVATEPOOL_PROJECT_ID: the Google Cloud project where your private pool is located.REGION: the region where you created your private pool.
Comments
Post a Comment