nova instance有3种状态：power_state, vm_state, task_state，分别对应horizon界面上的Power State，Status，Task
Openstack wiki上有介绍：
- power_state is the hypervisor state, loaded “bottom-up” from compute worker;
- vm_state reflects the stable state based on API calls, matching user expectation, revised “top-down” within API implementation.
- task_state reflects the transition state introduced by in-progress API calls.
- power_state and vm_state may conflict with each other, which needs to be resolved case-by-case.
Power_state
power_state反映的是hypervisor的状态。目前都是用libvirt作为hypervisor。
例如，nova会调用libvirt api去查询instance的state
[root@node-2 ~]# virsh list --all
Id Name State
----------------------------------------------------
229 instance-00000139 running
230 instance-00000138 running
231 instance-00000137 running
232 instance-00000155 running
- instance-00000136 shut off
- instance-0000013a shut off
[root@node-2 ~]# virsh domstate instance-00000136
shut off在libvirt中，共有几种状态：
- undefined
- defined(也叫stopped)
- running
- paused
- saved
具体解释在这里
下图表示在几种状态的转换关系：
Notes：
- 有的转换依赖于domain（instance）是transient domain还是persistent domain， 如running ---(shutdown)---> defined/undefined。两者的区别在这里。思考：Openstack创建的domain是否都是persistent domain？
-
virsh create创建transient domain；virsh define;virsh start创建、启动persistent domain
How is it updated？
- 调hypervisor api取得的状态总是对的；如果nova数据库中的状态与其不一致，那么更新nova数据库。
- nova有定时任务来检查、更新power_state
- 如果对instance做了操作（task），并且该操作可能影响power_state，那么在该task结束之前需要更新power_state
power_state list
目前（2015/9/21 master）有6种状态：
# nova/compute/power_state.py
# NOTE(maoy): These are *not* virDomainState values from libvirt.
# The hex value happens to match virDomainState for backward-compatibility
# reasons.
NOSTATE = 0x00 RUNNING = 0x01 PAUSED = 0x03 SHUTDOWN = 0x04 # the VM is powered off CRASHED = 0x06 SUSPENDED = 0x07 # TODO(justinsb): Power state really needs to be a proper class, # so that we're not locked into the libvirt status codes and can put mapping # logic here rather than spread throughout the code STATE_MAP = { NOSTATE: 'pending', RUNNING: 'running', PAUSED: 'paused', SHUTDOWN: 'shutdown', CRASHED: 'crashed', SUSPENDED: 'suspended', }
Vm_state
vm_state反映基于API调用的一种稳定状态，而task_state反映的是一种瞬时状态。
How is it updated？
- 只有当一个task（一个compute api call）结束时，才能更新vm_state。
- 如果成功则同时设置task_state=None，如果失败，vm_state可能不变（task支持rollback），也可能设置为ERROR。
vm_state list
# nova/compute/vm_states.py
ACTIVE = 'active' # VM is running
BUILDING = 'building' # VM only exists in DB PAUSED = 'paused' SUSPENDED = 'suspended' # VM is suspended to disk. STOPPED = 'stopped' # VM is powered off, the disk image is still there. RESCUED = 'rescued' # A rescue image is running with the original VM image attached. RESIZED = 'resized' # a VM with the new size is active. The user is expected to manually confirm or revert. SOFT_DELETED = 'soft-delete' # VM is marked as deleted but the disk images are still available to restore. DELETED = 'deleted' # VM is permanently deleted. ERROR = 'error' SHELVED = 'shelved' # VM is powered off, resources still on hypervisor SHELVED_OFFLOADED = 'shelved_offloaded' # VM and associated resources are not on hypervisor ALLOW_SOFT_REBOOT = [ACTIVE] # states we can soft reboot from ALLOW_HARD_REBOOT = ALLOW_SOFT_REBOOT + [STOPPED, PAUSED, SUSPENDED, ERROR] # states we allow hard reboot from
Task_state
task_state反映了基于API调用的瞬时状态。
它表示instance当前处于某个API调用中的某个阶段，比vm_state更细化，让用户了解某个action/task执行到哪个步骤了，比如创建instance时会有scheduling -> block_device_mapping -> networking -> spawning。但调用完之后设置为None。
task_state list
# nova/compute/task_states.py
"""Possible task states for instances.
urrent moment. These tasks can be generic, such as 'spawning', or specific,
such as 'block_device_mapping'. These task states allow for a better view into
what an instance is doing and should be displayed to users/administrators as
necessary. """ # possible task states during create() SCHEDULING = 'scheduling' BLOCK_DEVICE_MAPPING = 'block_device_mapping' NETWORKING = 'networking' SPAWNING = 'spawning' # possible task states during snapshot() IMAGE_SNAPSHOT = 'image_snapshot' IMAGE_SNAPSHOT_PENDING = 'image_snapshot_pending' IMAGE_PENDING_UPLOAD = 'image_pending_upload' IMAGE_UPLOADING = 'image_uploading' # possible task states during backup() IMAGE_BACKUP = 'image_backup' # possible task states during set_admin_password() UPDATING_PASSWORD = 'updating_password' # possible task states during resize() RESIZE_PREP = 'resize_prep' RESIZE_MIGRATING = 'resize_migrating' RESIZE_MIGRATED = 'resize_migrated' RESIZE_FINISH = 'resize_finish' # possible task states during revert_resize() RESIZE_REVERTING = 'resize_reverting' # possible task states during confirm_resize() RESIZE_CONFIRMING = 'resize_confirming' # possible task states during reboot() REBOOTING = 'rebooting' REBOOT_PENDING = 'reboot_pending' REBOOT_STARTED = 'reboot_started' REBOOTING_HARD = 'rebooting_hard' REBOOT_PENDING_HARD = 'reboot_pending_hard' REBOOT_STARTED_HARD = 'reboot_started_hard' # possible task states during pause() PAUSING = 'pausing' # possible task states during unpause() UNPAUSING = 'unpausing' # possible task states during suspend() SUSPENDING = 'suspending' # possible task states during resume() RESUMING = 'resuming' # possible task states during power_off() POWERING_OFF = 'powering-off' # possible task states during power_on() POWERING_ON = 'powering-on' # possible task states during rescue() RESCUING = 'rescuing' # possible task states during unrescue() UNRESCUING = 'unrescuing' # possible task states during rebuild() REBUILDING = 'rebuilding' REBUILD_BLOCK_DEVICE_MAPPING = "rebuild_block_device_mapping" REBUILD_SPAWNING = 'rebuild_spawning' # possible task states during live_migrate() MIGRATING = "migrating" # possible task states during delete() DELETING = 'deleting' # possible task states during soft_delete() SOFT_DELETING = 'soft-deleting' # possible task states during restore() RESTORING = 'restoring' # possible task states during shelve() SHELVING = 'shelving' SHELVING_IMAGE_PENDING_UPLOAD = 'shelving_image_pending_upload' SHELVING_IMAGE_UPLOADING = 'shelving_image_uploading' # possible task states during shelve_offload() SHELVING_OFFLOADING = 'shelving_offloading' # possible task states during unshelve() UNSHELVING = 'unshelving'
task_state state machine diagram (in-progress):
https://docs.google.com/spreadsheets/d/1uvrFI_L86_tBcZGlE2ck3RnMgtsgjIqYVtWTIT9eD8I/edit#gid=3
http://docs.openstack.org/developer/nova/devref/vmstates.html
http://docs.openstack.org/developer/nova/vmstates.html
Code walk-through
nova-compute定义了如下文件：
- nova/compute/power_state.py
- nova/compute/vm_states.py
- nova/compute/task_states.py
nova-api从自身角度定义了vm_state和task_state的对应关系
- nova/api/openstack/common.py
from nova.compute import task_states
from nova.compute import vm_states _STATE_MAP = { vm_states.ACTIVE: { 'default': 'ACTIVE', task_states.REBOOTING: 'REBOOT', # 前面是task_state, 后面是显示在cli/horizon上的'Status' task_states.REBOOT_PENDING: 'REBOOT', task_states.REBOOT_STARTED: 'REBOOT', task_states.REBOOTING_HARD: 'HARD_REBOOT', task_states.REBOOT_PENDING_HARD: 'HARD_REBOOT', task_states.REBOOT_STARTED_HARD: 'HARD_REBOOT', task_states.UPDATING_PASSWORD: 'PASSWORD', task_states.REBUILDING: 'REBUILD', task_states.REBUILD_BLOCK_DEVICE_MAPPING: 'REBUILD', task_states.REBUILD_SPAWNING: 'REBUILD', task_states.MIGRATING: 'MIGRATING', task_states.RESIZE_PREP: 'RESIZE', task_states.RESIZE_MIGRATING: 'RESIZE', task_states.RESIZE_MIGRATED: 'RESIZE', task_states.RESIZE_FINISH: 'RESIZE', }, vm_states.BUILDING: { 'default': 'BUILD', }, vm_states.STOPPED: { 'default': 'SHUTOFF', task_states.RESIZE_PREP: 'RESIZE', task_states.RESIZE_MIGRATING: 'RESIZE', task_states.RESIZE_MIGRATED: 'RESIZE', task_states.RESIZE_FINISH: 'RESIZE', task_states.REBUILDING: 'REBUILD', task_states.REBUILD_BLOCK_DEVICE_MAPPING: 'REBUILD', task_states.REBUILD_SPAWNING: 'REBUILD', }, vm_states.RESIZED: { 'default': 'VERIFY_RESIZE', # Note(maoy): the OS API spec 1.1 doesn't have CONFIRMING_RESIZE # state so we comment that out for future reference only. #task_states.RESIZE_CONFIRMING: 'CONFIRMING_RESIZE', task_states.RESIZE_REVERTING: 'REVERT_RESIZE', }, vm_states.PAUSED: { 'default': 'PAUSED', task_states.MIGRATING: 'MIGRATING', }, vm_states.SUSPENDED: { 'default': 'SUSPENDED', }, vm_states.RESCUED: { 'default': 'RESCUE', }, vm_states.ERROR: { 'default': 'ERROR', task_states.REBUILDING: 'REBUILD', task_states.REBUILD_BLOCK_DEVICE_MAPPING: 'REBUILD', task_states.REBUILD_SPAWNING: 'REBUILD', }, vm_states.DELETED: { 'default': 'DELETED', }, vm_states.SOFT_DELETED: { 'default': 'SOFT_DELETED', }, vm_states.SHELVED: { 'default': 'SHELVED', }, vm_states.SHELVED_OFFLOADED: { 'default': 'SHELVED_OFFLOADED', }, } def status_from_state(vm_state, task_state='default'): """Given vm_state and task_state, return a status string.""" task_map = _STATE_MAP.get(vm_state, dict(default='UNKNOWN')) status = task_map.get(task_state, task_map['default']) if status == "UNKNOWN": LOG.error(_LE("status is UNKNOWN from vm_state=%(vm_state)s " "task_state=%(task_state)s. Bad upgrade or db " "corrupted?"), {'vm_state': vm_state, 'task_state': task_state}) return status
从status_from_state方法看，_STATE_MAP表示vm_state处于某个状态时，所有可能的task_state有哪些，都穷举出来了。这里注意，某些task_state的default值，在nova/compute/task_states.py中没有定义，如：
vm_states.ACTIVE: {
'default': 'ACTIVE',
...}
vm_states.STOPPED: {
'default': 'SHUTOFF',
...}
vm_states.RESIZED: {
'default': 'VERIFY_RESIZE',
...}
vm_states.ERROR: {
'default': 'ERROR',
...}给定一对vm_state/task_state，status_from_state能返回一个status字符串，这个status就是cli/horizon上的'Status'：
[root@node-129 ~]# nova list | cut -d"|" -f1-6 +--------------------------------------+-------------------------------------------------------+---------+------------+-------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ID | Name | Status | Task State | Power State +--------------------------------------+-------------------------------------------------------+---------+------------+-------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | f6d4e67b-885c-4a10-891f-d91675e52bae | 123 | SHUTOFF | - | Shutdown | 05be07c9-6b2a-4db9-b03b-8322c035816c | 652 | SHUTOFF | - | Shutdown | 19d1308c-f19b-4860-979f-220fd376ef58 | ITMGMT | SHUTOFF | - | Shutdown | c496c710-29f8-46c3-90f8-6c2606d4b252 | Winboo-vm | ACTIVE | - | Running | a27f4030-ece5-4aeb-a84e-70bf0389f81f | abc | ACTIVE | - | Running
比较一下，nova-api列出的"Status"其实比nova-compute的vm_state（14个）多，共20个：
- ACTIVE
- REBOOT
- HARD_REBOOT
- PASSWORD
- REBUILD
- MIGRATING
- RESIZE
- BUILD
- SHUTOFF
- VERIFY_RESIZE
- REVERT_RESIZE
- PAUSED
- SUSPENDED
- RESCUE**
- ERROR
- DELETED
- SOFT_DELETED**
- SHELVED
- SHELVED_OFFLOADED**
- UNKNOWN
状态转移
参考:
- http://docs.openstack.org/developer/nova/devref/vmstates.html
- http://docs.openstack.org/developer/nova/vmstates.html
不同的操作，只有当instance处于特定的vm_state和task_state时才能进行。这个控制不是由nova-api来完成的，而是由nova-compute来决定的：nova/compute/api.py定义了nova-compute暴露给外部的接口。大多数方法名称就是操作名，如create,restore, shelve, soft_delete。
基本上所有方法都用check_instance_state修饰了。这个修饰器限定了该操作在某些vm_state/task_state下才可以进行。比如：
@wrap_check_policy
@check_instance_lock
@check_instance_state(vm_state=[vm_states.SOFT_DELETED]) def restore(self, context, instance): """Restore a previously deleted (but not reclaimed) instance.""" @wrap_check_policy @check_instance_cell @check_instance_state(vm_state=[vm_states.ACTIVE, vm_states.STOPPED, vm_states.PAUSED, vm_states.SUSPENDED]) def snapshot(self, context, instance, name, extra_properties=None): """Snapshot the given instance.""" @wrap_check_policy @check_instance_lock @check_instance_state(vm_state=set( vm_states.ALLOW_SOFT_REBOOT + vm_states.ALLOW_HARD_REBOOT), task_state=[None, task_states.REBOOTING, task_states.REBOOT_PENDING, task_states.REBOOT_STARTED, task_states.REBOOTING_HARD, task_states.RESUMING, task_states.UNPAUSING, task_states.PAUSING, task_states.SUSPENDING]) def reboot(self, context, instance, reboot_type): """Reboot the given instance."""
所以只要查看check_instance_state的参数就能确定操作可行与否。
