YugabyteDB Anywhere Terraform Provider
Terraform を使用して AWS に YugabyteDB Anywhere と Universe をデプロイします
Yugabyte Anywhere Terraform Provider は、YugabyteDB Anywhere (YBA) のリソースを管理するための Terraform Provider です。 この記事では Terraform を使用して、AWS の複数リージョンにまたがる YugabyteDB をデプロイする手順を紹介します。
必要なもの
この記事の手順で必要なものは次のとおりです。
- AWS Account
- Terraform 1.8.x
- YBA のライセンスファイル
YBA は有償プロダクトのため、ライセンスが必要です。
AWS リソースの準備
YBA をセットアップして Universe(≒ クラスター)を AWS の複数リージョンにまたがってデプロイするためには、事前に次のリソースを準備する必要があります。
- YBA をデプロイする VPC と Subnet
- ノードをデプロイする VPC と Subnet(リージョンごと)
- YBA と各リージョンのノード間の VPC Peering
- 各リージョンのノード間の VPC Peering
- YBA の Security Group
- ノードの Security Group(リージョンごと)
- YBA の EC2 にアタッチする IAM Role
- ノードの EC2 にアタッチする IAM Role
- YBA の Key Pair
- YBA をインストールする EC2
今回は 3 つのリージョンにまたがって Universe を作成するため、次のように Provider を定義します。 Tokyo (ap-northeast-1) と N. Virginia (us-east-1) と London (eu-west-2) を使ってみることにします。
provider "aws" {
alias = "ap-northeast-1"
region = "ap-northeast-1"
}
provider "aws" {
alias = "us-east-1"
region = "us-east-1"
}
provider "aws" {
alias = "eu-west-2"
region = "eu-west-2"
}
YBA をデプロイするための VPC を作成します。
module "yba_vpc" {
source = "terraform-aws-modules/vpc/aws"
providers = {
aws = aws.ap-northeast-1
}
name = "yba"
cidr = "10.0.0.0/16"
azs = ["ap-northeast-1a", "ap-northeast-1c", "ap-northeast-1d"]
public_subnets = ["10.1.0.0/24", "10.1.1.0/24", "10.1.2.0/24"]
}
Universe をデプロイするための VPC を作成します。
module "cluster_vpc_1" {
source = "terraform-aws-modules/vpc/aws"
providers = {
aws = aws.ap-northeast-1
}
name = "yb-cluster-1"
cidr = "10.1.0.0/16"
azs = ["ap-northeast-1a", "ap-northeast-1c", "ap-northeast-1d"]
public_subnets = ["10.1.0.0/24", "10.1.1.0/24", "10.1.2.0/24"]
}
同じように us-east-1 と eu-west-2 にも VPC を作成します。あとで VPC Peering を作成するため、VPC の CIDR は被らないようにします。
次に YBA ↔ ノード間の VPC Peering を作成します。
resource "aws_vpc_peering_connection" "yba_cluster_1" {
provider = aws.ap-northeast-1
vpc_id = module.yba_vpc.vpc_id
peer_vpc_id = module.cluster_vpc_1.vpc_id
peer_region = "ap-northeast-1"
auto_accept = false
}
resource "aws_vpc_peering_connection_accepter" "yba_cluster_1" {
provider = aws.ap-northeast-1
vpc_peering_connection_id = aws_vpc_peering_connection.yba_cluster_1.id
auto_accept = true
}
resource "aws_vpc_peering_connection_options" "yba_cluster_1" {
provider = aws.ap-northeast-1
vpc_peering_connection_id = local.yba_cluster_1_vpc_peering_connection_id
accepter {
allow_remote_vpc_dns_resolution = true
}
}
resource "aws_route" "yba_cluster_1" {
provider = aws.ap-northeast-1
for_each = toset(module.yba_vpc.public_route_table_ids)
route_table_id = each.key
destination_cidr_block = module.cluster_vpc_1.vpc_cidr_block
vpc_peering_connection_id = aws_vpc_peering_connection.yba_cluster_1.id
}
resource "aws_route" "cluster_1_yba" {
provider = aws.ap-northeast-1
for_each = toset(module.cluster_vpc_1.public_route_table_ids)
route_table_id = each.key
destination_cidr_block = module.yba_vpc.vpc_cidr_block
vpc_peering_connection_id = aws_vpc_peering_connection.yba_cluster_1.id
}
3 つのリージョンのノード間の VPC Peering を作成します。
resource "aws_vpc_peering_connection" "cluster_1_cluster_2" {
provider = aws.ap-northeast-1
vpc_id = module.cluster_vpc_1.vpc_id
peer_vpc_id = module.cluster_vpc_2.vpc_id
peer_region = "us-east-1"
auto_accept = false
}
resource "aws_vpc_peering_connection_accepter" "cluster_1_cluster_2" {
provider = aws.us-east-1
vpc_peering_connection_id = aws_vpc_peering_connection.cluster_1_cluster_2.id
auto_accept = true
}
resource "aws_vpc_peering_connection_options" "cluster_1_cluster_2" {
provider = aws.us-east-1
vpc_peering_connection_id = local.cluster_1_cluster_2_vpc_peering_connection_id
accepter {
allow_remote_vpc_dns_resolution = true
}
}
resource "aws_route" "cluster_1_cluster_2" {
provider = aws.ap-northeast-1
for_each = toset(module.cluster_vpc_1.public_route_table_ids)
route_table_id = each.key
destination_cidr_block = module.cluster_vpc_2.vpc_cidr_block
vpc_peering_connection_id = aws_vpc_peering_connection.cluster_1_cluster_2.id
}
resource "aws_route" "cluster_2_cluster_1" {
provider = aws.us-east-1
for_each = toset(module.cluster_vpc_2.public_route_table_ids)
route_table_id = each.key
destination_cidr_block = module.cluster_vpc_1.vpc_cidr_block
vpc_peering_connection_id = aws_vpc_peering_connection.cluster_1_cluster_2.id
}
同じように us-east-1 ↔ eu-west-2 と eu-west-2 ↔ ap-northeast-1 の VPC Peering も作成します。
次に YBA の Security Group を作成します。VPC Peering を利用した通信なので、すべてのポートを許可していますが、細かく制御したい場合は、次の URL に YBA の通信要件の記載があります。
YugabyteDB Anywhere networking requirements | YugabyteDB Docs
module "yba_security_group" {
source = "terraform-aws-modules/security-group/aws"
providers = {
aws = aws.ap-northeast-1
}
name = "yba"
description = "yba"
vpc_id = module.yba_vpc.vpc_id
ingress_cidr_blocks = [
module.yba_vpc.vpc_cidr_block,
module.cluster_vpc_1.vpc_cidr_block,
module.cluster_vpc_2.vpc_cidr_block,
module.cluster_vpc_3.vpc_cidr_block
]
ingress_rules = ["all-all"]
egress_rules = ["all-all"]
}
ノードの Security Group を作成します。
module "cluster_1_security_group" {
source = "terraform-aws-modules/security-group/aws"
providers = {
aws = aws.ap-northeast-1
}
name = "yb-cluster-1"
description = "yb-cluster-1"
vpc_id = module.cluster_vpc_1.vpc_id
ingress_cidr_blocks = [
module.yba_vpc.vpc_cidr_block,
module.cluster_vpc_1.vpc_cidr_block,
module.cluster_vpc_2.vpc_cidr_block,
module.cluster_vpc_3.vpc_cidr_block,
]
ingress_rules = ["all-all"]
egress_rules = ["all-all"]
}
同じように us-east-1 と eu-west-2 にも Security Group を作成します。
次に YBA の EC2 にアタッチする IAM Policy を作成します。 YBA は EC2 を動的に作成するため、EC2 に関連する Permission が必要になります。
module "yba_policy" {
source = "terraform-aws-modules/iam/aws//modules/iam-policy"
name_prefix = "yba-"
path = "/"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"iam:PassRole",
"ec2:AttachVolume",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:ImportVolume",
"ec2:ModifyVolumeAttribute",
"ec2:DescribeInstances",
"ec2:DescribeInstanceAttribute",
"ec2:CreateKeyPair",
"ec2:DescribeVolumesModifications",
"ec2:DeleteVolume",
"ec2:DescribeVolumeStatus",
"ec2:StartInstances",
"ec2:DescribeAvailabilityZones",
"ec2:DescribeVolumes",
"ec2:ModifyInstanceAttribute",
"ec2:DescribeKeyPairs",
"ec2:DescribeInstanceStatus",
"ec2:DetachVolume",
"ec2:ModifyVolume",
"ec2:TerminateInstances",
"ec2:AssignIpv6Addresses",
"ec2:ImportKeyPair",
"ec2:DescribeTags",
"ec2:CreateTags",
"ec2:RunInstances",
"ec2:AssignPrivateIpAddresses",
"ec2:StopInstances",
"ec2:AllocateAddress",
"ec2:DescribeVolumeAttribute",
"ec2:DescribeSecurityGroups",
"ec2:CreateVolume",
"ec2:EnableVolumeIO",
"ec2:DescribeImages",
"ec2:DescribeVpcs",
"ec2:DeleteSecurityGroup",
"ec2:DescribeSubnets",
"ec2:DeleteKeyPair",
"ec2:DescribeVpcPeeringConnections",
"ec2:DescribeRouteTables",
"ec2:DescribeInternetGateways",
"ec2:GetConsoleOutput",
"ec2:CreateSnapshot",
"ec2:DeleteSnapshot",
"ec2:DescribeInstanceTypes"
],
"Resource": "*"
}
]
}
EOF
}
ノードの EC2 にアタッチする IAM Role を作成します。
module "node_role" {
source = "terraform-aws-modules/iam/aws//modules/iam-assumable-role"
create_role = true
create_instance_profile = true
role_name = "yb-node"
role_requires_mfa = false
custom_role_policy_arns = [
"arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore",
"arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy"
]
trusted_role_services = [
"ec2.amazonaws.com"
]
}
YBA の EC2 に設定する Key Pair を作成します。
module "yba_key_pair" {
source = "terraform-aws-modules/key-pair/aws"
providers = {
aws = aws.ap-northeast-1
}
key_name = "yba"
public_key = file("path/to/.ssh/yba_rsa.pub")
}
YBA をインストールする EC2 を作成します。
data "aws_ssm_parameter" "amazonlinux_2023" {
name = "/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64"
}
# https://docs.yugabyte.com/preview/yugabyte-platform/prepare/server-yba/
# YugabyteDB Anywhere doesn't support ARM architectures (but can be used to deploy universes to ARM-based nodes).
module "yba_ec2" {
source = "terraform-aws-modules/ec2-instance/aws"
providers = {
aws = aws.ap-northeast-1
}
name = "yba"
instance_type = "t3.xlarge" # YBA requires 4 CPUs
ami = data.aws_ssm_parameter.amazonlinux_2023.value
key_name = module.yba_key_pair.key_pair_name
vpc_security_group_ids = [module.yba_security_group.security_group_id]
subnet_id = module.yba_vpc.public_subnets[0]
associate_public_ip_address = true
create_iam_instance_profile = true
iam_role_policies = {
YBAPolicy = module.yba_policy.arn
AmazonSSMManagedInstanceCore = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
CloudWatchAgentServerPolicy = "arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy"
}
root_block_device = [
{
encrypted = true
volume_type = "gp3"
volume_size = 250 # YBA requires 200 GiB
}
]
}
YBA のインストール
さて、ようやくここで、YugabyteDB Anywhere Terraform Provider の登場です。
provider.yba
に YBA の IP アドレスを設定します。
provider "yba" {
alias = "unauthenticated"
host = "<YBA_IP_ADDRESS>"
}
resource.yba_installer
を使用すると、EC2 に対して YBA のインストールを実行できます。
resource "yba_installer" "install" {
provider = yba.unauthenticated
ssh_host_ip = "<YBA_IP_ADDRESS>"
ssh_user = "ec2-user"
ssh_private_key_file_path = "path/to/.ssh/yba_rsa"
yba_license_file = "path/to/yba_license.lic"
yba_version = "2.20.4.0-b50"
}
インストールが完了したら YBA の Web Console (https://<YBA IP ADDRESS>) にアクセスしてユーザーを登録します。
Universe の作成
新しく provider.yba
を定義します。api_token
は YBA Web Console で生成します。
provider "yba" {
host = "<YBA_IP_ADDRESS>"
api_token = "<API_TOKEN>"
}
Cloud Provider を作成します。
resource "yba_cloud_provider" "aws" {
code = "aws"
name = "aws-provider"
aws_config_settings {
use_iam_instance_profile = true
}
regions {
code = "ap-northeast-1"
name = "ap-northeast-1"
security_group_id = "<SECURITY_GROUP_ID>"
vnet_name = "<VPC_ID>"
yb_image = "<AMI_ID>"
zones {
code = "ap-northeast-1a"
name = "ap-northeast-1a"
subnet = "<SUBNET_ID>"
}
zones {
code = "ap-northeast-1c"
name = "ap-northeast-1c"
subnet = "<SUBNET_ID>"
}
zones {
code = "ap-northeast-1d"
name = "ap-northeast-1d"
subnet = "<SUBNET_ID>"
}
}
regions {
code = "us-east-1"
name = "us-east-1"
security_group_id = "<SECURITY_GROUP_ID>"
vnet_name = "<VPC_ID>"
yb_image = "<AMI_ID>"
zones {
code = "us-east-1a"
name = "us-east-1a"
subnet = "<SUBNET_ID>"
}
zones {
code = "us-east-1c"
name = "us-east-1c"
subnet = "<SUBNET_ID>"
}
zones {
code = "us-east-1d"
name = "us-east-1d"
subnet = "<SUBNET_ID>"
}
}
regions {
code = "eu-west-2"
name = "eu-west-2"
security_group_id = "<SECURITY_GROUP_ID>"
vnet_name = "<VPC_ID>"
yb_image = "<AMI_ID>"
zones {
code = "eu-west-2a"
name = "eu-west-2a"
subnet = "<SUBNET_ID>"
}
zones {
code = "eu-west-2b"
name = "eu-west-2b"
subnet = "<SUBNET_ID>"
}
zones {
code = "eu-west-2c"
name = "eu-west-2c"
subnet = "<SUBNET_ID>"
}
}
ssh_port = 22
air_gap_install = false
}
resource.yba_universe
を定義して Universe を作成します。
data "yba_provider_key" "aws" {
provider_id = yba_cloud_provider.aws.id
}
data "yba_release_version" "release_version" {
// To fetch default YBDB version
}
resource "yba_universe" "demo" {
clusters {
cluster_type = "PRIMARY"
user_intent {
universe_name = "demo"
provider_type = yba_cloud_provider.aws.code
provider = yba_cloud_provider.aws.id
region_list = yba_cloud_provider.aws.regions[*].uuid
num_nodes = 3
replication_factor = 3
instance_type = "c5.large"
assign_public_ip = true
aws_arn_string = "<INSTANCE_PROFILE_ARN>"
device_info {
num_volumes = 1
volume_size = 40
storage_type = "GP3"
disk_iops = 3000
throughput = 125
}
use_time_sync = true
enable_ysql = true
yb_software_version = data.yba_release_version.release_version.id
access_key_code = data.yba_provider_key.aws.id
}
}
}
YBA の Web Console で進捗を確認してみましょう。 最後のステップまで完了すれば Multi-region Universe の完成です 🎉🎉