资源编排配合实例自定义数据，实现RDS自动创建与恢复-低调大师

资源编排配合实例自定义数据，实现RDS自动创建与恢复

2018-09-16 666

背景：不少客户需要批量部署系统，他们希望ROS不仅能帮助他们自动部署底层PAAS与IAAS资源，还能够省去安装软件，链接数据库，导入数据库文件的动作。而ROS能完美结合ECS的自定义数据，自定义镜像，RDS Web API，最大化减少人肉工作。

前提条件

a.拥有可用的OSS空间（Bucket）,且与创建的RDS同一个Region

b.RDS具有访问OSS的权限（也可通过ROS授权）

c.了解ROS，可通过ROS控制台创建模板和资源

步骤概览：

a.创建ECS自定义镜像，镜像内包含自己的应用软件

b.将需要备份恢复的数据库.bak（本文以SQLServer为例）文件上传到OSS空间

c.编写ROS的模板，自动创建所需资源，创建顺序如下VPC>Vswitch>SecurityGroup>RDS>ECS，ECS创建成功后会自动执行ROS模板里面编写的自定义数据，连接数据库，并且将OSS中的备份文件恢复到RDS里面。

步骤详解：

1.创建自定义镜像

1.1创建一个ECS实例，部署好所需的软件，除了安装客户业务所需要的软件之外，为了实现自动的备份，减少配置工作，也需要安装一些阿里云SDK

1.1.1安装python

请根据Python官网的引导安装合适的Python版本，推荐安装2.7.10。

1.1.2安装完毕后，查看Python版本。

如果你是Windows操作系统：

执行C:\>c:\Python27\python.exe -V查看Python版本，如果输出内容为：Python 2.7.10表明您已成功安装了Python 2.7.10版本。

如果提示不是内部或外部命令，请检查配置Path环境变量，增加Python的安装路径和pip命令的目录，如下图所示：

如果你是Mac/Linux/Unix操作系统：

执行$ python -V查看Python版本，如果输出内容为：Python 2.7.10表明您已成功安装了Python 2.7.10版本。

1.1.3安装SDK依赖包

使用pip安装或者git clone源码安装，任选其一。

Pip安装

pip install aliyun-python-sdk-rds

pip install oss2

源码安装

# git 克隆openapi

git clone https://github.com/aliyun/aliyun-openapi-python-sdk.git

# 安装阿里云 SDK 核心库

cd aliyun-python-sdk-core

python setup.py install

# 安装阿里云 RDS SDK

cd aliyun-python-sdk-rds

python setup.py install

# git 克隆OSS SDK

git clone https://github.com/aliyun/aliyun-oss-python-sdk.git

cd aliyun-oss-python-sdk

# 安装OSS2

python setup.py install

1.1.4 下载备份恢复的.py执行脚本

下载实例级别迁移上云脚本RDSSQLCreateMigrateTasksBatchly.py，下载地址：单击下载。保存在目录中，本次假设保存在C盘。

1.2控制台中将上一步骤设置好的ECS生成自定义镜像，详细参考创建自定义镜像。

注意：自定义镜像属于Region，不能跨Region直接读取，如果需要跨账号或跨Region使用，请参考共享镜像。

1.2.1 记录自定义镜像的ID与所在的Region

2.将备份文件，即数据库.bak文件上传到对应的OSS Bucket上（本次以SQLServer为例），并且记录下对应的OSSd的Endpoint和url地址

3.编写ROS的模板

3.1前两个步骤已经将基本的环境准备好，现在就需要编写ROS模板，让ROS自动创建所需资源，然后通过ROS给ECS指派的自定义数据，自动运行.py的备份脚本，

3.2在执行RDSSQLCreateMigrateTasksBatchly.py文件时，需要了解使用说明，如下

~/Downloads/RDSSQLCreateMigrateTasksBatchly.py -k <access_key_id> -s <access_key_secret> -i <rds_instance_id> -e <oss_endpoint> -b <oss_bucket> -d <directory>

参数说明：

参数	说明
access_key_id	阿里云账号对应的access key id
access_key_secret	阿里云账号对应的access key secret
rds_instance_id	RDS SQL Server目标实例ID
oss_endpoint	备份文件所在的OSS Bucket endpoint地址，获取方法请参见OSS Endpoint错误中截图
oss_bucket	备份文件所在的OSS Bucket名字
directory	OSS Bucket中，备份文件所在的目录，如果是根目录，请传入“/”。

因此，在ROS中，除了oss_bucket和directory是指定的，其他参数均可通过ROS的Fn::GetAtt函数获得，

因此，对应的ECS的自定义数据Userdata如下

"UserData": {

          "Fn::Join": [

            "",

            [

              "[powershell]",

              "\n",

              "$keyid=",

              {

                "Fn::GetAtt": [

                   "RamAK",

                  "AccessKeyId"

                ]

              },

              "\n",

              "$keysec=",

              {

                "Fn::GetAtt": [

                  "RamAK",

                  "AccessKeySecret"

                ]

              },

              "\n",

              "$rdsid=",

              {

                "Fn::GetAtt": [

                  "LYDatabase",

                  "DBInstanceId"

                ]

              },

              "\n",

              "python C:\\RDSSQLCreateMigrateTasksBatchly.py -k '$keyid -s '$keysec -i '$rdsid -e oss_endpoint -b oss_bucket -d directory"

            ]

          ]

        }

如下为userdata最终执行的示例：

python ~/Downloads/RDSSQLCreateMigrateTasksBatchly.py -k LTAIQazXKPRwwErT -s BMkIUhroubQOLpOMqfA09IKlqp4G2k -i rm-2zesz5774ud8s71i5 -e oss-cn-beijing.aliyuncs.com -b atp-test-on-ecs -d Migration/OPENAPIDemo

注意：上面执行所需要的Accesskey ID和Accesskey Secret均为RAM账号，无法使用主账号，所以如果您在阿里云的主账号下运行，请先创建一个RAM账号，并赋予RDS的完整管理权限，创建RAM账号与赋予RDS权限语句如下

{

  "ROSTemplateFormatVersion": "2015-09-01",

  "Description": "",

  "Parameters": {

    "UserName": {

      "AllowedPattern": "^[a-zA-Z0-9.@-_]+$",

      "ConstraintDescription": "[1, 64]. Consist of lowercase letter, number, '@', '.', '_' or '-'",

      "Default":"lionel",

      "Description": "The name of sub account",

      "Type": "String"

    },

    "Password": {

      "ConstraintDescription": "[8, 30] Consist of uppercase letter, lowercase letter and number.",

      "Description": "The login password of sub account",

      "MaxLength": 32,

      "MinLength": 8,

      "Default":"lionelpassword",

      "Confirm": true,

      "Type": "String"

    }

  },

  "Mappings": {},

  "Conditions": {},

  "Resources": {

    "RamAK": {

      "Properties": {

        "UserName": {

          "Fn::GetAtt": [

            "RamUser",

            "UserName"

          ]

        }

      },

      "Type": "ALIYUN::RAM::AccessKey"

    },

    "RamUser": {

      "Properties": {

        "LoginProfile": {

          "Password": {

            "Ref": "Password"

          },

          "PasswordResetRequired": false

        },

        "UserName": {

          "Ref": "UserName"

        }

      },

      "Type": "ALIYUN::RAM::User"

},

"RDSfullaccessPolicy": {

         "Type": "ALIYUN::RAM::ManagedPolicy",

         "Properties": {

           "Description":  "Description",

           "PolicyName":"AliyunRDSfullacessPolicy",

           "PolicyDocument":  {

               "Version": "1",

               "Statement":[{

                   "Action":

                    [ "rds:*"]

                   ,

                   "Resource": ["*"],

                   "Effect": "Allow"

                 },

                 {

                     "Action":

                       ["dms:*LoginDatabase"]

                     ,

                     "Resource": ["acs:rds:*:*:*"],

                     "Effect": "Allow"

                   }

               ]

             },

           "Users":  [ { "Ref": "RamUser" } ]

         }

       }

3.3若要RDS能读取OSS里面的.bak备份文件，则需要给RDS授权读取OSS，可以通过在控制台手动赋予权限，亦可以通过ROS自动化实现，实现Json语言如下

"Policy": {

   "Type": "ALIYUN::RAM::ManagedPolicy",

   "Properties": {

     "Description": "Description",

     "PolicyName": "AliyunRDSImportPolicy",

     "PolicyDocument": {

       "Version": "1",

       "Statement": [

         {

           "Action": [

             "oss:GetObject",

             "oss:GetBucketLocation"

           ],

           "Resource": [

             "*"

           ],

           "Effect": "Allow"

         }

       ]

     },

     "Roles": [

       {

         "Ref": "ImportRole"

       }

     ]

   }

 },

 "ImportRole": {

  "Type": "ALIYUN::RAM::Role",

  "Properties": {

    "RoleName": "AliyunRDSImportRole",

    "Description": "RDS import",

    "AssumeRolePolicyDocument": {

      "Version": "1",

      "Statement": [

        {

          "Action": "sts:AssumeRole",

          "Effect": "Allow",

          "Principal": {

            "Service": [

              "rds.aliyuncs.com"

            ]

          }

        }

      ]

    }

  }

}

3.4因为ECS里面的.py脚本需要在RDS彻底创建成功后才能执行，所以要通过ROS DependsOn参数,让ECS在RDS生成后创建

"Server": {

      "Type": "ALIYUN::ECS::Instance",

      "DependsOn": [

        "LYDatabase"

      ],

      "Properties": {

        "IoOptimized": "optimized",

        "ZoneId": "cn-shenzhen-b",

        "SystemDiskSize": "100",

        "InstanceChargeType": "PostPaid",

        "SecurityGroupId": {

          "Ref": "ServerSecurityGroup"

        },

        "VSwitchId": {

          "Ref": "ServerSwitch"

        },

        "Period": 1,

        "SystemDiskCategory": "cloud_efficiency",

        "InternetChargeType": "PayByTraffic",

        "InternetMaxBandwidthOut": 1,

        "VpcId": {

          "Ref": "ALIYUN-ECS-VPC1"

        },

        "InternetMaxBandwidthIn": 200,

        "ImageId": "m-bp19l9u9broshpsresc9",

        "InstanceType": "ecs.sn1ne.xlarge",

        "AllocatePublicIP": true,

        "HostName": "lyServer",

        "Password": "lypass!@#",

"UserData": {}

4.总结：通过如上的过程即可实现数据库的自动部署和自动备份恢复，完成的ROS代码参考如下。

{
  "ROSTemplateFormatVersion": "2015-09-01",
  "Description": "",
  "Parameters": {
    "UserName": {
      "AllowedPattern": "^[a-zA-Z0-9.@-_]+$",
      "ConstraintDescription": "[1, 64]. Consist of lowercase letter, number, '@', '.', '_' or '-'",
      "Default":"lionel",
      "Description": "The name of sub account",
      "Type": "String"
    },
    "Password": {
      "ConstraintDescription": "[8, 30] Consist of uppercase letter, lowercase letter and number.",
      "Description": "The login password of sub account",
      "MaxLength": 32,
      "MinLength": 8,
      "Default":"lionelpassword",
      "Confirm": true,
      "Type": "String"
    }
  },
  "Mappings": {},
  "Conditions": {},
  "Resources": {
    "RamAK": {
      "Properties": {
        "UserName": {
          "Fn::GetAtt": [
            "RamUser",
            "UserName"
          ]
        }
      },
      "Type": "ALIYUN::RAM::AccessKey"
    },
    "RamUser": {
      "Properties": {
        "LoginProfile": {
          "Password": {
            "Ref": "Password"
          },
          "PasswordResetRequired": false
        },
        "UserName": {
          "Ref": "UserName"
        }
      },
      "Type": "ALIYUN::RAM::User"
    },
    "ImportRole": {
      "Type": "ALIYUN::RAM::Role",
      "Properties": {
        "RoleName": "AliyunRDSImportRole",
        "Description": "RDS import",
        "AssumeRolePolicyDocument":{
           "Version": "1",
            "Statement": [{
              "Effect": "Allow",
              "Principal": {
                "Service": [ "rds.aliyuncs.com"]
              },
                "Action": "sts:AssumeRole"
              }]
            }
       }
    },
    "Policy": {
         "Type": "ALIYUN::RAM::ManagedPolicy",
         "Properties": {
           "Description":  "Description",
           "PolicyName":"AliyunRDSImportPolicy",
           "PolicyDocument":  {
               "Version": "1",
               "Statement":[{
                   "Action": [
                     "oss:GetObject",
                     "oss:GetBucketLocation"
                   ],
                   "Resource": ["*"],
                   "Effect": "Allow"
                 }]
             },
           "Roles":  [ { "Ref": "ImportRole" } ]
         }
       },
    "RDSfullaccessPolicy": {
         "Type": "ALIYUN::RAM::ManagedPolicy",
         "Properties": {
           "Description":  "Description",
           "PolicyName":"AliyunRDSfullacessPolicy",
           "PolicyDocument":  {
               "Version": "1",
               "Statement":[{
                   "Action":
                    [ "rds:*"]
                   ,
                   "Resource": ["*"],
                   "Effect": "Allow"
                 },
                 {
                     "Action":
                       ["dms:*LoginDatabase"]
                     ,
                     "Resource": ["acs:rds:*:*:*"],
                     "Effect": "Allow"
                   }
               ]
             },
           "Users":  [ { "Ref": "RamUser" } ]
         }
       },
    "ALIYUN-ECS-VPC1": {
      "Metadata": {
        "GraphicId": "RosGraphicElement-1"
      },
      "Type": "ALIYUN::ECS::VPC",
      "Properties": {
        "CidrBlock": "172.16.0.0/12"
      }
    },
    "ServerSecurityGroup": {
      "Type": "ALIYUN::ECS::SecurityGroup",
      "Properties": {
        "VpcId": {
          "Ref": "ALIYUN-ECS-VPC1"
        },
        "SecurityGroupIngress": [
          {
            "SourceGroupOwnerId": "",
            "SourceCidrIp": "0.0.0.0/0",
            "PortRange": "80/80",
            "SecurityGroupId": "mySecurityGroup",
            "NicType": "internet",
            "SourceGroupOwnerAccount": "",
            "Priority": 1,
            "SourceGroupId": "",
            "Policy": "accept",
            "IpProtocol": "tcp"
          },
          {
            "SourceGroupOwnerId": "",
            "SourceCidrIp": "0.0.0.0/0",
            "PortRange": "3389/3389",
            "SecurityGroupId": "mySecurityGroup",
            "NicType": "internet",
            "SourceGroupOwnerAccount": "",
            "Priority": 1,
            "SourceGroupId": "",
            "Policy": "accept",
            "IpProtocol": "tcp"
          },
          {
            "SourceGroupOwnerId": "",
            "SourceCidrIp": "0.0.0.0/0",
            "PortRange": "443/443",
            "SecurityGroupId": "mySecurityGroup",
            "NicType": "internet",
            "SourceGroupOwnerAccount": "",
            "Priority": 1,
            "SourceGroupId": "",
            "Policy": "accept",
            "IpProtocol": "tcp"
          }
        ]
      }
    },
    "ServerSwitch": {
      "Type": "ALIYUN::ECS::VSwitch",
      "Properties": {
        "VpcId": {
          "Ref": "ALIYUN-ECS-VPC1"
        },
        "CidrBlock": "172.16.1.0/24",
        "ZoneId": "cn-shenzhen-a"
      }
    },
    "LYDatabase": {
      "Type": "ALIYUN::RDS::DBInstance",
      "Properties": {
        "MasterUsername": "lyadmin",
        "MasterUserPassword": "lypasswrd",
        "Engine": "SQLServer",
        "EngineVersion": "2016_web",
        "MasterUserType":"Super",
        "VpcId": {
          "Ref": "ALIYUN-ECS-VPC1"
        },
        "ZoneId": "cn-shenzhen-a",
        "DBInstanceNetType": "Intranet",
        "DBInstanceClass": "mssql.x2.medium.w1",
        "SecurityIPList": "0.0.0.0/0",
        "DBInstanceStorage": "20",
        "VSwitchId": {
          "Ref": "ServerSwitch"
        },
        "ConnectionMode": "Performance",
        "PreferredBackupTime": "00:00Z-01:00Z",
        "BackupRetentionPeriod": 7,
        "PreferredBackupPeriod": [
          "Sunday"
        ]
      }
    },
    "Server": {
      "Type": "ALIYUN::ECS::Instance",
      "DependsOn": [
        "LYDatabase"
      ],
      "Properties": {
        "SystemDiskCategory": "cloud_efficiency",
        "IoOptimized": "optimized",
        "InternetChargeType": "PayByTraffic",
        "VpcId": {
          "Ref": "ALIYUN-ECS-VPC1"
        },
        "HostName": "Server",
        "ImageId": "win2016_64_dtc_1607_zh-cn_40G_alibase_20170915.vhd",
        "InstanceChargeType": "PostPaid",
        "VSwitchId": {
          "Ref": "ServerSwitch"
        },
        "Password": "Ly1pass!@#",
        "InstanceType": "ecs.n4.large",
        "SystemDiskSize": "40",
        "ZoneId": "cn-shenzhen-a",
        "InternetMaxBandwidthOut": 1,
        "InternetMaxBandwidthIn": 200,
        "UserData": {
          "Fn::Join": [
            "",
            [
              "[powershell]",
              "\n",
              "$keyid=",
              {
                "Fn::GetAtt": [
                   "RamAK",
                  "AccessKeyId"
                ]
              },
              "\n",
              "$keysec=",
              {
                "Fn::GetAtt": [
                  "RamAK",
                  "AccessKeySecret"
                ]
              },
              "\n",
              "$rdsid=",
              {
                "Fn::GetAtt": [
                  "LYDatabase",
                  "DBInstanceId"
                ]
              },
              "\n",
              "python C:\\RDSSQLCreateMigrateTasksBatchly.py -k '$keyid -s '$keysec -i '$rdsid -e oss_endpoint -b oss_bucket -d directory"
            ]
          ]
        },
        "SecurityGroupId": {
          "Ref": "ServerSecurityGroup"
        },
        "Period": 1,
        "AllocatePublicIP": true
      }
    }
  },
  "Outputs": {}
}

微信关注我们

原文链接：https://yq.aliyun.com/articles/640923

转载内容版权归作者及来源网站所有！

低调大师中文资讯倾力打造互联网数据资讯、行业资源、电子商务、移动互联网、网络营销平台。持续更新报道IT业界、互联网、市场资讯、驱动更新,是最及时权威的产业资讯及硬件资讯报道平台。

Kubernetes-使用rancher搭建Kubernetes高可用集群

引言之前我写了一篇文章介绍了离线部署Kubernetes 1.9.0，当时搭建的是1+3模式，此种模式不适合用于生产，为了防止由于master节点宕机造成集群不能正常工作的问题，推荐搭建Kubernetes高可用集群，本篇文章将介绍如何使用rancher搭建Kubernetes高可用集群。准备工作集群机器首先准备搭建集群环境的机器，笔者准备的集群机器如下： Hostname IP Memory OS Role k8smaster1 192.168.246.128 6G CentOS 7.5 Kubernetes master 节点 k8smaster2 192.168.246.129 6G CentOS 7.5 Kubernetes master 节点 k8smaster3 192.168.246.130 6G CentOS 7.5 Kubernetes master 节点 k8sworker1 192.168.246.131 12G CentOS 7.5 Kubernetes node 节点 k8sworker2 192.168.246.132 12G CentOS 7....

2018-09-17

668

竞价实例优化运营成本竞价实例(Spot Instance)也叫抢占式实例是一种按需实例，旨在降低部分场景下使用ECS的成本，创建竞价实例时，必须为指定的实例规格设置一个价格上限，当指定的实例规格的当前市场价格低于出价时，就能成功创建竞价实例，并按当前市场价格计费。默认能稳定持有实例一小时。之后，当市场价格高于出价或者资源供需关系变化时，实例会被自动释放。合理的使用阿里云ECS竞价实例，最高可以降低50% – 90% 的运营成本(相比按量付费的实例)，可以用相同的预算，将计算容量提升 2 – 10 倍。为了保证尽可能高概率的弹出竞价实例，可以设置多个AZ多个规格进行竞价，大大提升了竞价实例的创建成功率。竞价实例助力Cluster-Autoscaler弹性升级上周，阿里云容器服务发布了Cluster-Autoscaler的支持，初次

2018-09-17

729

资源下载

更多资源

优质分享App

近一个月的开发和优化，本站点的第一个app全新上线。该app采用极致压缩，本体才4.36MB。系统里面做了大量数据访问、缓存优化。方便用户在手机上查看文章。后续会推出HarmonyOS的适配版本。

腾讯云软件源

为解决软件依赖安装时官方源访问速度慢的问题，腾讯云为一些软件搭建了缓存服务。您可以通过使用腾讯云软件源站来提升依赖包的安装速度。为了方便用户自由搭建服务架构，目前腾讯云软件源站支持公网访问和内网访问。

Rocky Linux

Rocky Linux（中文名：洛基）是由Gregory Kurtzer于2020年12月发起的企业级Linux发行版，作为CentOS稳定版停止维护后与RHEL（Red Hat Enterprise Linux）完全兼容的开源替代方案，由社区拥有并管理，支持x86_64、aarch64等架构。其通过重新编译RHEL源代码提供长期稳定性，采用模块化包装和SELinux安全架构，默认包含GNOME桌面环境及XFS文件系统，支持十年生命周期更新。

Sublime Text

Sublime Text具有漂亮的用户界面和强大的功能，例如代码缩略图，Python的插件，代码段等。还可自定义键绑定，菜单和工具栏。Sublime Text 的主要功能包括：拼写检查，书签，完整的 Python API ， Goto 功能，即时项目切换，多选择，多窗口等等。Sublime Text 是一个跨平台的编辑器，同时支持Windows、Linux、Mac OS X等操作系统。