租户副本扩容-c7电子娱乐
本文为「让技术被看见 | 2024 oceanbase 布道师计划」的参赛文章。诚邀更多的技术爱好者参与 oceanbase 技术征文,赢取万元大奖,和我们一起用文字让代码跳动起来!
背景
生产上使用 黑屏命令行方式 创建租户时 ,有时候会因为疏忽只创建了单副本的租户,oceanbase 是分布式数据库,生产环境一般都会以多个zone来支撑高可用性,副本的数量由zone的数量来决定,集群中存在多少个zone,就可以有多少个副本 。生产环境的单副本租户会存在严重的风险隐患,本文浅谈单幅本租户如何扩容为多副本。副本类型说明可参见c7电子娱乐官网,本文不做详细介绍
集群信息
版本:4.2.1.8
集群规模:1-1-1(三个zone,每个zone有一个observer)
租户类型:mysql租户
通过 dba_ob_zones 视图可以看到,集群中存在三个zone: zone1 zone2 zone3,可以支撑租户存在三个全功能副本
mysql [oceanbase]> select * from dba_ob_zones;
------- ---------------------------- ---------------------------- -------- ----- -------- -----------
| zone | create_time | modify_time | status | idc | region | type |
------- ---------------------------- ---------------------------- -------- ----- -------- -----------
| zone1 | 2024-12-06 20:44:36.306251 | 2024-12-06 20:45:12.546557 | active | hz | zj | readwrite |
| zone2 | 2024-12-06 20:44:36.307335 | 2024-12-06 20:45:12.560635 | active | hz | zj | readwrite |
| zone3 | 2024-12-06 20:44:36.307335 | 2024-12-06 20:45:12.584586 | active | hz | zj | readwrite |
------- ---------------------------- ---------------------------- -------- ----- -------- -----------
3 rows in set (0.200 sec)
mysql [oceanbase]>
租户副本信息
通过 dba_ob_tenants 视图的 locality字段值 和 unt_num可以看出,obmysql租户当前只在zone1上存在一个unit和一个全功能副本,不满足多副本的高可用特性。
mysql [oceanbase]> select * from oceanbase.dba_ob_tenants where tenant_name = 'obmysql'\g
*************************** 1. row ***************************
tenant_id: 1002
tenant_name: obmysql
tenant_type: user
create_time: 2024-12-06 20:56:08.766386
modify_time: 2024-12-06 20:56:32.286591
primary_zone: random
locality: full{1}@zone1
previous_locality: null
compatibility_mode: mysql
status: normal
in_recyclebin: no
locked: no
tenant_role: primary
switchover_status: normal
switchover_epoch: 0
sync_scn: 1733490519199984806
replayable_scn: 1733490519199984806
readable_scn: 1733490519199984806
recovery_until_scn: 4611686018427387903
log_mode: noarchivelog
arbitration_service_status: disabled
unit_num: 1
compatible: 4.2.1.8
max_ls_id: 1001
1 row in set (0.102 sec)
租户副本扩容
扩充资源池
c7电子娱乐官网声明 在增加副本之前,首先需要确认租户在目标 zone 上是否有资源池,如果没有,则需要调整现有资源池的 zone 列表,或者新增一个资源池给租户,如果新增资源池,要求资源池的 unit 个数与租户已有的资源池一致,这里选择修改资源池方法,修改资源池具体语法可参见c7电子娱乐官网
通过 dba_ob_tenants 视图取到租户的 tenant_id 为 1002
mysql [oceanbase]> select * from oceanbase.dba_ob_tenants where tenant_name = 'obmysql'\g
*************************** 1. row ***************************
tenant_id: 1002
tenant_name: obmysql
tenant_type: user
create_time: 2024-12-06 20:56:08.766386
modify_time: 2024-12-06 20:56:32.286591
primary_zone: random
locality: full{1}@zone1
previous_locality: null
compatibility_mode: mysql
status: normal
in_recyclebin: no
locked: no
tenant_role: primary
switchover_status: normal
switchover_epoch: 0
sync_scn: 1733490519199984806
replayable_scn: 1733490519199984806
readable_scn: 1733490519199984806
recovery_until_scn: 4611686018427387903
log_mode: noarchivelog
arbitration_service_status: disabled
unit_num: 1
compatible: 4.2.1.8
max_ls_id: 1001
1 row in set (0.102 sec)
根据tenant_id到dba_ob_resource_pools视图查看 zone_list 值得知:当前obmysql租户的资源池只在zone1上存在,zone2 与 zone3上均不存在
mysql [oceanbase]> select * from dba_ob_resource_pools where tenant_id=1002\g
*************************** 1. row ***************************
resource_pool_id: 1001
name: pool_obmysql_zone1_qem
tenant_id: 1002
create_time: 2024-12-06 20:56:08.651042
modify_time: 2024-12-06 20:56:08.774144
unit_count: 1
unit_config_id: 1001
zone_list: zone1
replica_type: full
1 row in set (0.006 sec)
mysql [oceanbase]>
扩充资源池至 zone2 zone3
mysql [oceanbase]> alter resource pool pool_obmysql_zone1_qem zone_list=('zone1','zone2','zone3');
query ok, 0 rows affected (0.687 sec)
mysql [oceanbase]> select * from dba_ob_resource_pools where tenant_id=1002\g
*************************** 1. row ***************************
resource_pool_id: 1001
name: pool_obmysql_zone1_qem
tenant_id: 1002
create_time: 2024-12-06 20:56:08.651042
modify_time: 2024-12-06 21:49:12.095918
unit_count: 1
unit_config_id: 1001
zone_list: zone1;zone2;zone3
replica_type: full
1 row in set (0.007 sec)
mysql [oceanbase]>
扩容副本
可以发现 不支持一次性将obmysql租户单副本扩容成三副本,必须先将副本扩容至第两个zone,再扩容为第三个zone,以此方式来达到三副本扩容。
mysql [oceanbase]> alter tenant obmysql locality="full{1}@zone1, full{1}@zone2, full{1}@zone3";
error 4179 (hy000): violate locality principal not allowed
mysql [oceanbase]>
mysql [oceanbase]> alter tenant obmysql locality="full{1}@zone1, full{1}@zone2";
query ok, 0 rows affected (0.190 sec)
mysql [oceanbase]>
查看副本扩容任务
通过 job_status 字段值 为 inprogress,可以看到扩容副本任务正在进行中
mysql [oceanbase]> select * from oceanbase.dba_ob_tenant_jobs where job_type = 'alter_tenant_locality'\g
*************************** 1. row ***************************
job_id: 2
job_type: alter_tenant_locality
job_status: inprogress
result_code: null
progress: 0
start_time: 2024-12-06 21:50:57.183389
modify_time: 2024-12-06 21:50:57.183389
tenant_id: 1002
sql_text: alter tenant obmysql locality="full{1}@zone1, full{1}@zone2"
extra_info: from: 'full{1}@zone1', to: 'full{1}@zone1, full{1}@zone2'
rs_svr_ip: 10.0.0.68
rs_svr_port: 2882
1 row in set (0.001 sec)
mysql [oceanbase]>
job_status 字段值 为 success ,表示扩容副本任务已经成功
mysql [oceanbase]> select * from oceanbase.dba_ob_tenant_jobs where job_type = 'alter_tenant_locality'\g
*************************** 1. row ***************************
job_id: 2
job_type: alter_tenant_locality
job_status: success
result_code: 0
progress: 100
start_time: 2024-12-06 21:50:57.183389
modify_time: 2024-12-06 21:51:28.246024
tenant_id: 1002
sql_text: alter tenant obmysql locality="full{1}@zone1, full{1}@zone2"
extra_info: from: 'full{1}@zone1', to: 'full{1}@zone1, full{1}@zone2'
rs_svr_ip: 10.0.0.68
rs_svr_port: 2882
1 row in set (0.007 sec)
租户副本成功扩容至两个副本之后,此时可以进行第三个副本扩容
mysql [oceanbase]> alter tenant obmysql locality="full{1}@zone1, full{1}@zone2, full{1}@zone3";
query ok, 0 rows affected (0.195 sec)
mysql [oceanbase]>
mysql [oceanbase]>
mysql [oceanbase]>
通过 dba_ob_tenant_jobs的job_status 字段值 为 success ,表示三副本任务已经成功
mysql [oceanbase]> select * from oceanbase.dba_ob_tenant_jobs where job_type = 'alter_tenant_locality'\g
*************************** 1. row ***************************
job_id: 2
job_type: alter_tenant_locality
job_status: success
result_code: 0
progress: 100
start_time: 2024-12-06 21:50:57.183389
modify_time: 2024-12-06 21:51:28.246024
tenant_id: 1002
sql_text: alter tenant obmysql locality="full{1}@zone1, full{1}@zone2"
extra_info: from: 'full{1}@zone1', to: 'full{1}@zone1, full{1}@zone2'
rs_svr_ip: 10.0.0.68
rs_svr_port: 2882
*************************** 2. row ***************************
job_id: 3
job_type: alter_tenant_locality
job_status: success
result_code: 0
progress: 100
start_time: 2024-12-06 21:59:35.171558
modify_time: 2024-12-06 22:00:06.430095
tenant_id: 1002
sql_text: alter tenant obmysql locality="full{1}@zone1, full{1}@zone2, full{1}@zone3"
extra_info: from: 'full{1}@zone1, full{1}@zone2', to: 'full{1}@zone1, full{1}@zone2, full{1}@zone3'
rs_svr_ip: 10.0.0.68
rs_svr_port: 2882
2 rows in set (0.003 sec)
mysql [oceanbase]>
副本扩容验证
查看 dba_ob_tenants 视图的 locality值 可以发现 obmysql租户的副本已经扩容为三副本
mysql [oceanbase]> select * from oceanbase.dba_ob_tenants where tenant_name = 'obmysql'\g
*************************** 1. row ***************************
tenant_id: 1002
tenant_name: obmysql
tenant_type: user
create_time: 2024-12-06 20:56:08.766386
modify_time: 2024-12-06 21:59:45.810128
primary_zone: random
locality: full{1}@zone1, full{1}@zone2, full{1}@zone3
previous_locality: null
compatibility_mode: mysql
status: normal
in_recyclebin: no
locked: no
tenant_role: primary
switchover_status: normal
switchover_epoch: 0
sync_scn: 1733493740356216339
replayable_scn: 1733493740356216339
readable_scn: 1733493740356216338
recovery_until_scn: 4611686018427387903
log_mode: noarchivelog
arbitration_service_status: disabled
unit_num: 1
compatible: 4.2.1.8
max_ls_id: 1004
1 row in set (0.115 sec)
mysql [oceanbase]>
通过ocp的租户基本信息查看 obmysql租户在三个zone上均存在 全功能副本,租户副本扩容成功
总结
1.生产环境命令行创建租户时要注意副本的设置,避免出现单幅本租户的情况
2.租户部分扩容前,要先扩容资源池,方可进行租户副本扩容。
3.租户单副本扩容时,应先扩容至两副本后,再扩容至三副本,不可一次性扩容成三副本