Rolling Out Updates to Managed Instance Groups

A managed instance group contains one or more virtual machine instances that are controlled using an instance template. To update instances in a managed instance group, you can make update requests to the group as a whole, using the Managed Instance Group Updater feature.

To learn more about instance groups, read the Instance Groups Overview.

The Managed Instance Group Updater allows you to easily deploy new versions of software to instances in your managed instance groups, while controlling the speed of deployment, the level of disruption to your service, and the scope of the update. The Updater offers two primary advantages:

• The rollout of an update happens automatically to your specifications, without the need for additional user input after the initial request.
• You can perform partial rollouts which allows for canary testing.

By allowing new software to be deployed inside an existing managed instance group, there is no need for you to reconfigure the instance group or reconnect load balancing, autoscaling, or autohealing each time a new version of software is rolled out. Without the Updater, new software versions must be deployed either by creating a new managed instance group with a new software version, requiring additional set up each time, or through a manual, user-initiated, instance-by-instance recreate. Both of these approaches require significant manual steps throughout the process.

Before you begin

• If you want to use the command-line examples in this guide:
  1. Install or update to the latest version of the gcloud command-line tool.
  2. Set a default region and zone.
• You have a managed instance group.
• You have an instance template you want to update your instance group to.

Starting a basic rolling update

A rolling update is an update that is gradually applied to all instances in an instance group until all instances have been updated. You can control various aspects of a rolling update, such as how many instances can be taken offline for the update, how long to wait between updating instances, whether the update affects all or just a portion of instances, and so on.

Here are things to keep in mind when making a rolling update:

• Updates are intent based. When you make the initial update request, the API returns a successful response to confirm that the request was valid but that does not indicate that the update was successful. You must check the status of the managed instance group to determine if your update was deployed successfully.

• The Instance Group Updater API is a declarative API. The API expects a request to specify the desired post update configuration of the managed instance group, rather than an explicit function call.

• The Updater feature supports up to two instance template versions in your managed instance group. This means that you can specify two different instance template versions for your managed instance group, which is useful for performing canary updates.

To start a basic rolling update where the update is applied to 100% of the instances in the group, follow the instructions below.

gcloud compute instance-groups managed rolling-action start-update [INSTANCE_GROUP] \
    --version template=[INSTANCE_TEMPLATE] [--zone [ZONE] | --region [REGION]]

https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/instanceGroupManagers/[MANAGED_INSTANCE_GROUP_NAME]

{
  "instanceTemplate": "global/instanceTemplates/example-template",
  "updatePolicy": {
    "type": "proactive"
   }
 }

Configuring options for your update

For more complex updates, you can configure additional options for a specific update request. These options are described below.

Max surge

Set the maxSurge property to allow the Updater to temporarily create new instances above the targetSize during the update. For example, if you set maxSurge to 5, the managed instance group will create up to 5 new instances above your target size, with the new instance template. Setting a higher maxSurge value speeds up your update, at the cost of additional instances, which are billed according to the Compute Engine price sheet.

If you do not set the maxSurge value, the default value is used. For zonal managed instance groups, the default for maxSurge is 1. For regional managed instance groups, the default is [NUMBER_OF_ZONES], where [NUMBER_OF_ZONES] is the number of zones associated with the regional managed instance group.

This option is recognized only when configured with the REPLACE minimal action, but is not supported with the RESTART action setting. You can specify either a fixed number, or specify a percentage if the managed instance group has 10 or more instances. If you set a percentage, the Updater rounds up the number of instances if necessary.

maxSurge only works if you have enough quota or resources to support the additional resources.

Max unavailable

Set the maxUnavailable configuration so that only a certain number of instances are unavailable at any time during the update. For example, if you set maxUnavailable to 5, then only 5 instances will be taken offline for updating at a time. Use this parameter to control how disruptive the update is to your service and to control the rate at which the update is deployed.

This number also includes any instances that are unavailable for other reasons. For example, if the instance group is in the process of being resized up, instances in the middle of being created might be unavailable; these instances would count towards the maxUnavailable number. You can specify either a fixed number, or specify a percentage if the managed instance group has 10 or more instances. If you set a percentage, the Updater rounds down the number of instances, if necessary.

The default value for maxUnavailable in a zonal managed instance group is 1. In a regional managed instance group, the default is [NUMBER_OF_ZONES], the number of zones selected for the regional managed instance group, by default, 3.

Minimum wait time

Set the minReadySeconds to specify the amount of time to wait before considering a newly created or restarted instance as updated. Use this feature to control the rate at which the update is deployed. The timer starts when both of the following conditions are satisfied:

• The instance's status is RUNNING.
• If health checking is enabled, when the health check returns HEALTHY.

Note that for the health check to return healthy, the Updater will:

1. Wait for up to the period of time specified by autohealingPolicies.initialDelaySec for the health check to return as HEALTHY.
2. Then, waits for the period of time specified by minReadySeconds.

If the health check does not return HEALTHY within the initialDelaySec, then the Updater declares the VM instance as unhealthy and potentially stops the update. While the VM instance is waiting for verification during the initialDelaySec and the minReadySeconds time period, the instance's currentAction is VERIFYING. However, the underlying VM instance status remains RUNNING.

If there are no health checks for the instance group, then the timer starts when the instance's status is RUNNING. The maximum value for the minReadySeconds property is 3600 seconds (1 hour).

Minimal action

Set the minimal action property to control the minimum action that Updater must perform to update the instances in the group. For example, if you set REPLACE as the minimum action, then all instances that are affected will be deleted and replaced with a new instance, whether or not it is necessary.

Setting a minimal action guarantees that the Updater will perform that action, at a minimum. However, if the Updater determines that the minimal action you specify is not enough to perform the update, it might perform a more disruptive action. For example, if you set RESTART as the minimal action, the Updater attempts to restart instances to apply the update. However, the Updater will perform a more disruptive action if the update requires it. Changing an instance's OS, for example, cannot be done by restarting the instance so the Updater will replace the instances in the instance group with new VM instances.

Applicable actions are REPLACE or RESTART:

• RESTART: Restarts the instance (performs a stop and start request). Note that if your update request requires that the instance be replaced to pick up the changes (for example, changing the image would require that the instance is deleted and replaced), it will be forced to perform REPLACE.

• REPLACE: Deletes the existing instance and creates a new instance from the target template. The Updater creates a brand new instance with all new instance properties, such as new internal and external IP addresses.

The following diagram visualizes how these options affect your instances.

Additional update examples

Here are some command-line examples with common configuration options.

Perform a rolling update of all virtual machine instances, but create up to 5 new instances above the target size at a time

gcloud compute instance-groups managed rolling-action start-update [INSTANCE_GROUP_NAME] \
    --version template=[NEW_TEMPLATE] --max-surge 5 [--zone [ZONE] | --region [REGION]]

Perform a rolling update with at most 3 unavailable machines and a minimum wait time of 3 minutes before marking a new instance as available

gcloud compute instance-groups managed rolling-action start-update [INSTANCE_GROUP_NAME] \
    --version template=[NEW_TEMPLATE] --min-ready 3m \
    --max-unavailable 3 [--zone [ZONE] | --region [REGION]]

For example, if you have 1000 instances and you ran this command, the Updater will create up to 100 new instances before it starts to remove instances running the previous instance template.

Perform a rolling update of all virtual machine instances, but create up to 10% new instances above the target size at a time

gcloud compute instance-groups managed rolling-action start-update [INSTANCE_GROUP_NAME] \
    --version template=[NEW_TEMPLATE] --max-surge 10% [--zone [ZONE] | --region [REGION]]

Starting a canary update

The Instance Group Updater feature allows you to perform canary updates, so that you can test your updates on a random subset of instances before fully committing to the update.

A canary update is an update that is applied to a partial number of instances in the instance group. Canary updates let you test new features or upgrades on a subset of instances, instead of rolling out a potentially disruptive update to all your instances. If an update is not going well, you only need to roll back a small number of instances, minimizing the disruption for your users. From the perspective of the server, a canary update is the same as a standard rolling update, except that the number of instances that should be updated is less than the total size of the instance group. Like a standard rolling update, a canary update is disruptive to the instances affected; that is, the affected instances are deleted and replaced by new VM instances during the update.

To initiate a canary update:

• You specify up to two instance template versions, typically a new instance template to canary and the current instance template for the remainder of the instances. For example, you can specify that 20% of your instances are created based on new-instance-template while the rest of the instances continue to run on old-instance-template. You cannot specify more than two instance templates at a time.

• You must always specify a target size (targetSize) for the canary version. You cannot initiate a canary update if you omit the target size for the canary version. For example, if you specified that 10% of instances should be used for canarying, the remaining 90% will be untouched and use the current instance template.

gcloud compute instance-groups managed rolling-action start-update [INSTANCE_GROUP_NAME] \
    --version template=[CURRENT_TEMPLATE] \
    --canary-version template=[NEW_TEMPLATE],target-size=[SIZE] \
    [--zone [ZONE] | --region [REGION]]

gcloud compute instance-groups managed rolling-action start-update my-ig1 \
        --version template=my-template-A --canary-version template=my-template-B,target-size=10%

https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/instanceGroupManagers/[INSTANCE_GROUP_NAME]

{
 "versions": [
  {
   "instanceTemplate": "global/instanceTemplates/[NEW_TEMPLATE]",
   "targetSize": {
    "[percent|fixed]": [NUMBER|PERCENTAGE] # Use `fixed` for a specific number of instances
   }
  },
  {
   "instanceTemplate": "global/instanceTemplates/[CURRENT_TEMPLATE]"
  }
 ]
}

Rolling forward a canary update

After running a canary update, you can decide if you want to commit the update to 100% of the instance group or roll back.

gcloud compute instance-groups managed rolling-action start-update [INSTANCE_GROUP_NAME] \
    --version template=[NEW_TEMPLATE] [--zone [ZONE] | --region [REGION]]

https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/instanceGroupManagers/[INSTANCE_GROUP_NAME]

{
"versions": [
   {
   "instanceTemplate": "global/instanceTemplates/[NEW_TEMPLATE]" # New instance template
   }
 ]
}

Starting an opportunistic or proactive update

By default, updates made using the gcloud command-line tool are proactive and updates initiated in the API are opportunistic.

For proactive updates, Compute Engine actively schedules actions to apply the requested updates to instances as necessary. In many cases, this often means deleting and recreating instances proactively.

Alternatively, you can choose to perform an opportunistic update, if a proactive update is potentially too disruptive. An opportunistic update is only applied when you manually initiate the update on selected instances or when new instances are created by the managed instance group. The managed instance group creates new instances when the managed instance group is resized either by another service, such as an autoscaler, or manually by a user. Compute Engine does not actively initiate requests to apply opportunistic updates.

In certain scenarios, an opportunistic update is useful because you don't want to cause instability to the system if it can be avoided. For example, if you have a non-critical update that can be applied as necessary without any urgency and you have a managed instance group that is actively being autoscaled, perform an opportunistic update so that Compute Engine does not actively tear down your existing instances to apply the update.

To choose whether an update is opportunistic or proactive, set the type property to OPPORTUNISTIC or PROACTIVE using either the gcloud command-line tool or the API.

gcloud compute instance-groups managed rolling-action start-update [INSTANCE_GROUP_NAME] \
    --version template=[INSTANCE_TEMPLATE] \
    --type [opportunistic|proactive] [--zone [ZONE] | --region [REGION]]

{
"updatePolicy": {
  "type": "PROACTIVE" # Performs a proactive update
},
"versions": [{
  "instanceTemplate": "global/instanceTemplates/[NEW_TEMPLATE]",
  }]
}

Updating selected instances

When you start an opportunistic update, you must wait for Compute Engine to roll out the update as opportunities arise. However, if you want more control over the rollout, you can manually initiate that update immediately on specific instances in your managed instance group.

Manually initiating updates allows you to:

• Select the instances that you would like to update.
• Apply an update with the least of amount of disruption that is necessary for the update to complete. For example, if you are only updating metadata, it might not be necessary to restart the instance to complete the update. When manually initiating the update, the minimal necessary action is automatically performed.
• Enforce instance restart or recreation even if those actions are not necessary for applying the update. For example, you might want to restart a VM even if you are only updating its metadata because your guest software has to pick up the new metadata on VM boot.
• Prevent an update if it requires more disruption than you can afford.
• Perform an update of all selected instances immediately, without restricting the rollout by maxSurge and maxUnavailable constraints.

Minimal action and most disruptive allowed action

Depending on the nature of an update, it might disrupt the instance's state. For example, changing an instance's machine type requires the VM to restart while changing its boot image requires deleting and replacing the instance.

You can control the level of disruption by using the minimal-action and most-disruptive-allowed-action flags:

• minimal-action allows you to force a more disruptive action than is necessary.
• most-disruptive-allowed-action allows you to prevent an update if it requires more disruption than you can afford.

When updating selected instances, both of these flags accept the following actions:

By default, the minimal-action is NONE. If your update requires a more disruptive action than you set with this flag, Compute Engine performs the necessary action to execute the update.

By default, the most-disruptive-allowed-action is REPLACE. If your update requires a more disruptive action than the actoin you set with this flag, the update request will fail. For example, if you set "restart" as the most disruptive allowed action, then an attempt to update the boot disk image will fail because that update requires instance recreation, a more disruptive action than a restart.

You can update selected instances with the gcloud tool or the API.

gcloud beta compute instance-groups managed update-instances [INSTANCE_GROUP] \
    --instances [INSTANCE_NAMES] \
    --most-disruptive-allowed-action [DISRUPTION_LEVEL] \
    --minimal-action [DISRUPTION_LEVEL]

POST https://www.googleapis.com/compute/beta/projects/[PROJECT]/regions/[REGION]/instanceGroupManagers/[INSTANCE_GROUP_NAME]/applyUpdatesToInstances
{
  "instances": "zones/[ZONE]/instances/[INSTANCE_NAME]","zones/[ZONE]/instances/[INSTANCE_NAME]"
  "minimalAction": [DISRUPTION_LEVEL],
  "mostDisruptiveAllowedAction": [DISRUPTION_LEVEL]
}

Performing a rolling replace or restart

Alternatively, you can use the restart or replace commands to perform a rolling restart or a rolling replace of VM instances in the managed instance group. Similar to the start-update command, you can specify any of the configuration options for a restart or a replace. A rolling restart or replace does not change anything about the instance group, including the instance template. It simply refreshes the instances in the group using the method you choose.

There are many reasons why you might want a rolling replace or a rolling restart. For example, you might want to refresh your VM instances from time to time to:

• Clear up memory leaks.
• Restart your application so it can run from a fresh machine.
• Force a periodic replace as a best practice to test your VMs.
• Update your VM's operating system image or rerun startup scripts to update your software.

To perform a replace where all instances are deleted and replaced with new instances:

gcloud compute instance-groups managed rolling-action replace [INSTANCE_GROUP]

gcloud compute instance-groups managed rolling-action restart [INSTANCE_GROUP]

PATCH https://www.googleapis.com/compute/v1/projects/myproject/zones/[ZONE]/instanceGroupManagers/[INSTANCE_GROUP_NAME]

{
 "updatePolicy": {
  "minimalAction": "REPLACE",
  "type": "PROACTIVE"
 },
 "versions": [
  {
   "instanceTemplate": "global/instanceTemplates/example-template",
   "name": "v2"
  }
 ]
}

PATCH https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/instanceGroupManagers/[INSTANCE_GROUP_NAME]

{
 "updatePolicy": {
  "minimalAction": "RESTART",
  "type": "PROACTIVE"
 },
 "versions": [
  {
   "instanceTemplate": "global/instanceTemplates/example-template",
   "name": "v2"
  }
 ]
}

Additional replace/restart examples

Perform a rolling restart of all virtual machines, two at a time

This command restarts all virtual machines in the instance group, two at a time. Notice that no new instance template is specified.

gcloud compute instance-groups managed rolling-action restart [INSTANCE_GROUP_NAME] \
    --max-unavailable 2 [--zone [ZONE] | --region [REGION]]

Rolling restart all VMs as quickly as possible

gcloud compute instance-groups managed rolling-action restart [INSTANCE_GROUP_NAME] \
    --max-unavailable 100% [--zone [ZONE] | --region [REGION]]

Rolling replace all VMs as quickly as possible

gcloud compute instance-groups managed rolling-action replace [INSTANCE_GROUP_NAME]  \
    --max-unavailable 100% [--zone [ZONE] | --region [REGION]]

Updating a regional managed instance group

A regional managed instance group contains virtual machine instances spread across multiple zones within a region, as opposed to zonal managed instance groups, which only contain instances in one zone. Regional managed instance groups give you the ability to distribute your instances across more than one zone to improve your application's availability and to support extreme cases where one zone fails or an entire group of instances stops responding.

Updating a regional managed instance group using the Instance Group Updater feature is same as performing an update on a zonal managed instance group, with a few exceptions described below. When you initiate an update to a regional instance group, the Updater always updates instances proportionally and evenly across each zone; it is not possible to choose which instances in which zones are updated first or to update instances in only one zone.

Differences between updating a regional versus zonal managed instance group

Before you initiate an update to a regional managed instance group, you should be aware that there are a number of things that behave differently than for zonal managed instance groups:

• The default update parameters for regional managed instance groups are maxUnavailable=[NUMBER_OF_ZONES] and maxSurge=[NUMBER_OF_ZONES], where [NUMBER_OF_ZONES] is the number of zones selected for the regional managed instance group, by default, 3.

• If you are using fixed numbers when specifying an update, the fixed number must be either 0 or equal to or greater than the number of zones associated with the regional managed instance group. For example, if the instance group is distributed across three zones, then you cannot set maxSurge to 1 or to 2 because the Updater needs to create an additional instance in each of the three zones.

Using a fixed number or a percentage in update requests

If you specify a fixed number in your update requests, the number you specify is divided by the number of zones in the regional managed instance group and distributed evenly. For example, if you specify maxSurge=10, then the Updater divides 10 across the number of zones in the region and create new instances based on that number. If the number of instances does not divide evenly across zones, the Updater adds the additional instances to a random zone. So, for 10 instances across three zones, two of the zones will get 3 instances and one zone will get 4 instances. The same logic is applied to the maxUnavailable and the targetSize parameter for canary updates.

You can only specify a percentage if your managed instance group contains 10 or more VM instances. Percentages are handled slightly differently depending on the situation:

• If you specify a percentage of VM instances for a canary update, the Updater attempts to distribute the instances evenly across zones. The remainder is rounded either up or down in each zone but the total difference will not be more than 1 VM instance per group. For example, for a managed instance group with 10 instances and a target size percentage of 25%, the update will be rolled out to 2 to 3 VM instances.

• If you specify a percentage for update options like maxSurge and maxUnavailable, the percentages are rounded independently per zone. The same rules that apply to updating zonal managed instance groups apply here.

Monitoring updates

After you initiate a rolling update, it can take some time for the update to finish. You can monitor the progress of an update by inspecting the status of the managed instance group, or by checking the currentAction on each instance in the managed instance group.

Group status

At the group level, Compute Engine populates a read-only field called status that contains a versionTarget.isReached flag and an isStable flag. You can use the gcloud tool or the API to access these flags.

To verify that your update to the instance group has finished rolling out, check that status.versionTarget.isReached==true. To verify that all instances in the group are running and healthy (that is, that the currentAction of each managed instance is NONE), check that status.isStable==true. Remember that the stability of a managed instance group depends on group configurations beyond the Updater; for example, if your group is autoscaled, and if it is currently scaling up, then isStable==false due to the autoscaler operation.

You can also use the console to see the current and target number of instances being updated.

gcloud compute instance-groups managed describe [INSTANCE_GROUP_NAME] \
    [--zone [ZONE] | --region [REGION]]

POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/instanceGroupManagers/[INSTANCE_GROUP_NAME]/get

status.versionTarget.isReached

The rollout of an update is considered complete when all VM instances in the group have been or are being created in their target version. In the case of a full rollout, all instances are configured to use the new instance template. In the case of a partial rollout, the instances are configured according to the specified target split between instance templates.

You can verify whether the rollout of an update is complete by checking the value of status.versionTarget.isReached field of an instanceGroupManagers (or regionInstanceGroupManagers) resource.

status.versionTarget.isReached is set to true if all VM instances have been or are being created using the target version (versions[]).

status.versionTarget.isReached is set to false when at least one VM is not yet using the target version (versions[]). Or, in the case of a canary update, status.versionTarget.isReached is set to false when the number of VMs using a target version (versions[].instanceTemplates) does not match its target size (versions[].targetSize).

You can also use the gcloud beta compute instance-groups managed wait-until command with the --version-target-reached flag to wait until status.versionTarget.isReached is set to true for the group:

gcloud beta compute instance-groups managed wait-until INSTANCE_GROUP_NAME \
    --version-target-reached \
    [--zone [ZONE] | --region [REGION]]

status.isStable

You can verify that a managed instance group is running and healthy by checking the value of the status.isStable field of the associated instanceGroupManagers or regionInstanceGroupManagers resource.

status.isStable set to false indicates that changes are active, pending, or that the managed instance group itself is being modified.

status.isStable set to true indicates the following:

• None of the instances in the managed instance group are undergoing any type of change and the currentAction for all instances is NONE.
• No changes are pending for instances in the managed instance group.
• The managed instance group itself is not being modified.

Managed instance groups can be modified in numerous ways. For example:

• You make a request to roll out a new instance template.
• You make a request to create, delete, resize or update instances in the group.
• An autoscaler requests to resize the group.
• An autohealer resource is replacing one or more unhealthy instances in the managed instance group.
• In a regional managed instance group, some of the instances are being redistributed.

As soon as all actions are finished, status.isStable is set to true again for that managed instance group.

You can also use the gcloud beta compute instance-groups managed wait-until command with the --stable flag, to wait until status.isStable is set to true for the group:

gcloud beta compute instance-groups managed wait-until INSTANCE_GROUP_NAME \
    --stable \
    [--zone [ZONE] | --region [REGION]]

Current actions on instances

To view current the currentAction being performed on each instance in your group, as well as the status of each instance, you can use the gcloud tool or the API.

gcloud compute instance-groups managed list-instances [INSTANCE_GROUP_NAME] [--filter="zone:([ZONE])" | --filter="region:([REGION])"]

NAME               ZONE           STATUS   ACTION    INSTANCE_TEMPLATE  VERSION_NAME  LAST_ERROR
vm-instances-9pk4  us-central1-f           CREATING  my-new-template
vm-instances-h2r1  us-central1-f           DELETING  my-old-template
vm-instances-j1h8  us-central1-f  RUNNING  NONE      my-old-template
vm-instances-ngod  us-central1-f  RUNNING  NONE      my-old-template

POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/instanceGroupManagers/[INSTANCE_GROUP_NAME]/listManagedInstances

{
 "managedInstances": [
  {
   "instance": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/instances/vm-instances-prvp",
   "id": "5317605642920955957",
   "instanceStatus": "RUNNING",
   "instanceTemplate": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/instanceTemplates/[INSTANCE_TEMPLATE_NAME]",
   "currentAction": "REFRESHING"
  },
  {
   "instance": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/instances/vm-instances-pz5j",
   "currentAction": "DELETING"
  },
  {
   "instance": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/instances/vm-instances-w2t5",
   "id": "2800161036826218547",
   "instanceStatus": "RUNNING",
   "instanceTemplate": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/instanceTemplates/[INSTANCE_TEMPLATE_NAME]",
   "currentAction": "REFRESHING"
  }
 ]
}

For each instance in a managed instance group, the status of the instance is described by its instanceStatus field. To see a list of valid instanceStatus fields, see the documentation for Checking Instance Status.

If the instance is undergoing some type of change, the currentAction field will also be populated with one of the following actions to help you track the progress of the change. Otherwise, the currentAction field will be NONE.

Possible currentAction values are:

• ABANDONING: The instance is being removed from the managed instance group.
• CREATING: The instance is in the process of being created.
• CREATING_WITHOUT_RETRIES: The instance is being created without retries; if the instance fails to be created on the first try, the managed instance group will not try to replace the instance again.
• DELETING: The instance is in the process of being deleted.
• RECREATING: The instance was deleted and is being replaced.
• REFRESHING: The instance is being removed from its current target pools and being readded to the list of current target pools (this list might be the same or different from existing target pools).
• RESTARTING: The instance is in the process of being restarted using the stop and start methods.
• VERIFYING: The instance has been created and is in the process of being verified.
• NONE: No actions are being performed on the instance.

When an instance is successfully updated, its instanceStatus field will reflect the instance's current state.

Rolling back an update

There is no explicit command for rolling back an update to a previous version but if you decide that you want to roll back an update (either a fully committed or canary update), you can do so by making a new update request and passing in the instance template that you want to roll back to.

For example, the following command rolls back an update as fast as possible:

gcloud compute instance-groups managed rolling-action start-update [INSTANCE_GROUP_NAME] \
    --version template=[OLD_INSTANCE_TEMPLATE] --max-unavailable 100% [--zone [ZONE] | --region [REGION]]

Replace [OLD_INSTANCE_TEMPLATE] with the name of the old instance template you want to roll back to.

In the API, make a PATCH request to the following URI:

https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/instanceGroupManagers/[INSTANCE_GROUP_NAME]

If the instance group is a regional managed instance group, replace zones/[ZONE] with regions/[REGION].

In the request body, specify the old instance template as a version:

{ "updatePolicy":
  {
    "maxUnavailable":
    {
      "percent": 100
    }
  },
 "versions": [
    {
      "instanceTemplate": "global/instanceTemplates/[OLD_TEMPLATE]" # Old instance template
    }
   ]
}

If it's a regional managed instance group with less than 10 instances, you must use a fixed value for maxUnavailable and set the value to the number of instances in the group:

{ "updatePolicy":
  {
    "maxUnavailable":
    {
      "fixed": [NUMBER_OF_INSTANCES_IN_GROUP]
    }
  },
 "versions": [
    {
      "instanceTemplate": "global/instanceTemplates/[OLD_TEMPLATE]" # Old instance template
    }
   ]
}

The Instance Group Updater service treats this as a regular update request so all of the update options described in this document can be specified with your request.

Controlling the speed of an update

By default, when you make an update request, the service performs the update as fast as possible. If you aren't sure you want to apply an update fully or are tentatively testing your changes, you can control the speed of the update using the following methods.

1. Start a canary update rather than a full update.
2. Set a large minReadySeconds value. Setting this value causes the service to wait this number of seconds before considering the instance successfully updated and proceeding to the next instance.
3. Enable health checking to cause the service to wait for your application to start and to report a healthy signal, before considering the instance successfully updated and proceeding to the next instance.
4. Set a low maxUnavailable and maxSurge value. This ensures that only a minimal number of instances are updated at a time.
5. Manually initiate updates on specific instances to update those instances immediately.

You can also use a combination of these parameters to control the rate of your update.

Stopping an update

There is no explicit method or command to stop an update. You can change an update from proactive to opportunistic and if the managed instance group is not being resized by other services like autoscaler, the change to opportunistic will effectively "stop" the update.

To change an update from proactive to opportunistic, run the following command:

gcloud compute instance-groups managed rolling-action stop-proactive-update [INSTANCE_GROUP_NAME] \
    [--zone [ZONE] | --region [REGION]]

If you decide you want to stop the update completely after converting it from proactive to opportunistic, you can stop it using these steps:

1. Make a request to determine how many instances have been updated.

   gcloud compute instance-groups managed list-instances [INSTANCE_GROUP_NAME] \
       [--zone [ZONE] | --region [REGION]]

   The gcloud tool returns a response with a list of instances in the instance group and their current statuses:

   NAME               ZONE           STATUS   ACTION    INSTANCE_TEMPLATE  VERSION_NAME  LAST_ERROR
   vm-instances-9pk4  us-central1-f  RUNNING  NONE      my-new-template
   vm-instances-j1h8  us-central1-f  RUNNING  NONE      my-new-template
   vm-instances-ngod  us-central1-f  RUNNING  NONE      my-old-template
   

   In this example, two instances have already been updated.

2. Next, make a request to perform a new "update" but pass in the number of instances that have already been updated as the target size:

   gcloud compute instance-groups managed rolling-action start-update [INSTANCE_GROUP_NAME] \
       --version template=my-old-template \
       --canary-version template=my-new-template,target-size=2 \
       [--zone [ZONE] | --region [REGION]]

   To the Updater service, this update appears "complete", so no other instances are updated, effectively stopping the update.

Relationship between versions and instanceTemplate properties for a managed instance group

Google recommends using the versions field to configure instance templates for managed instance groups.

The legacy instanceTemplate field overlaps in functionality with versions as both fields allow you to specify which instance template the managed instance group will use to create new instances. However, only the versions field allows you to specify an advanced, two template (canary) configuration.

For backwards compatibility, managed instance groups continue to support setting the top-level instanceTemplate field, even though we recommend that you switch to using only the versions field. Using both fields at the same time can lead to ambiguity and confusion.

If you specify both the instanceTemplate field and the versions field when calling the update() or patch() method, there are three possibilities of what could happen:

• You set both fields to the same value.

  This is a valid request. In this case, it creates no ambiguity and the new instance template is applied to the managed instance group.

  For example, in the following request, the top-level instanceTemplate and the versions field specify the same instance template which is different from the existing current template. The managed instance group will be updated to the new instance template:

  {
   "instanceTemplate": "global/instanceTemplates/new-template",
   "versions": [
    {
     "instanceTemplate": "global/instanceTemplates/new-template"
    }
   ],
   "updatePolicy": {
     "type": "PROACTIVE"
   }
  }
  

• You set both fields that do not match but only one value is different from the current instance template in the managed instance group.

  This is a valid request. The field that's different from the current setting is taken as the intended value. For example, you call the get() method, change one of the two fields and then call update() with just one changed field:

  {
   "instanceTemplate": "global/instanceTemplates/current-template",
   "versions": [
    {
     "instanceTemplate": "global/instanceTemplates/new-template"
    }
   ],
   "updatePolicy": {
     "type": "PROACTIVE"
   }
  }
  

• You set both fields but they don't match, have different values, and both are different from the current instance template in the managed instance group.

  This setting is invalid and will return an error as there is no clear intent.

  {
   "instanceTemplate": "global/instanceTemplates/new-template",
   "versions": [
    {
     "instanceTemplate": "global/instanceTemplates/a-different-new-template"
    }
   ],
   "updatePolicy": {
     "type": "PROACTIVE"
   }
  }
  

The versions field, instanceTemplate field, and the get() method

If you only specify one instance template, either through the top-level instanceTemplate field or through the versions field or through both, the get() method returns both fields in its response. This makes the new versions field backwards compatible. As long as you specify a single instance template in either of these fields, there is no change to what get() returns in the instanceTemplate field.

If the versions field has two instance templates specified, the get() method will return an empty top-level instanceTemplate field. There is no way to unambiguously express a canary, two instance template configuration in a the top-level instanceTemplate field so the field is not used during a canary update.

The versions field and the setInstanceTemplate() method

For backwards compatibility, the setInstanceTemplate() method behaves as it has previously, allowing you to change the template the managed instance group uses to create new instances. When you call this method, the versions field is overridden with the instance template specified by the setInstanceTemplate() method.

Furthermore, the method sets the updatePolicy to OPPORTUNISTIC. This prevents the managed instance group from actively deploying an instance template which was not explicitly specified in the versions field.

What's next

• Learn more about instance groups.
• Learn more about instance templates.