Centos下部署mongo切片

运维
字数统计: 2.7k   |   阅读时长≈ 14 分钟

为何需要水平分片

1 减少单机请求数，将单机负载，提高总负载
2 减少单机的存储空间，提高总存空间。

部署流程

第一步：所有服务器安装MongoDB—-all

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
vim /etc/yum.repos.d/mongodb-org.repo

		[mogodb-org]
		name=MongoDB Repository
		baseurl=http://mirrors.aliyun.com/mongodb/yum/redhat/7Server/mongodb-org/3.2/x86_64/
		gpgcheck=0
		enabled=1
		
yum clean all
yum makecache
yum install mongodb-org  -y
```  
第二步：配置configsvr----192.168.3.101
```bash	
#配置文件：config.conf

systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/config.log
storage:
  dbPath: /var/lib/mongo
  journal:
	enabled: true
processManagement:
  fork: true  # fork and run in background
  pidFilePath: /var/run/mongodb/config.pid  # location of pidfile
net:
  port: 27017
  bindIp: 192.168.3.101  # Listen to local interface only, comment to listen on all interfaces.
sharding:
  clusterRole: configsvr

	  
#启动mongo   
mongod -f /etc/mongodb/config.conf

第三步：配置shard—-192.168.3.102 192.168.3.103

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
#配置文件：shard1.conf

systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/shard.log
storage:
  dbPath: /var/lib/mongo
  journal:
	enabled: true
processManagement:
  fork: true  # fork and run in background
  pidFilePath: /var/run/mongodb/shard1.pid  # location of pidfile
net:
  port: 27017
  #bindIp: 127.0.0.1  # Listen to local interface only, comment to listen on all interfaces.
  bindIp: 192.168.3.102  # Listen to local interface only, comment to listen on all interfaces.
sharding:
  clusterRole: shardsvr

#配置文件：shard2.conf

systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/shard.log
storage:
  dbPath: /var/lib/mongo
  journal:
	enabled: true
processManagement:
  fork: true  # fork and run in background
  pidFilePath: /var/run/mongodb/shard1.pid  # location of pidfile
net:
  port: 27017
  #bindIp: 127.0.0.1  # Listen to local interface only, comment to listen on all interfaces.
  bindIp: 192.168.3.103  # Listen to local interface only, comment to listen on all interfaces.
sharding:
  clusterRole: shardsvr

#启动mongo  
mongod -f  /etc/mongodb/shard1.conf
mongod -f  /etc/mongodb/shard2.conf

第四步：配置route — 192.168.3.100

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
#配置文件： mongos.conf

systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/route.log
processManagement:
  fork: true  # fork and run in background
  pidFilePath: /var/run/mongodb/mongos.pid  # location of pidfile
net:
  port: 27017
  bindIp: 192.168.3.100  # Listen to local interface only, comment to listen on all interfaces.
#监听的配置服务器,只能有1个或者3个 configs为配置服务器的副本集名字
#configDB前面四个空格 不能用tab键
sharding:
    configDB: 192.168.3.101:27017

#启动服务器的mongos server   注：使用mongos启动，不是用mongod
mongos -f  /etc/mongodb/mongos.conf

通过./mongos –help命令可以查看启动路由相关参数信息。chunkSize为数据块大小，默认为200MB，为了便于测试这里将值设置为1

第五步：开启分片

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
#连接到路由实例后，可以通过sh.status()命令查看分片状态信息
	
	[root@localhost bin]# mongo   192.168.3.100          #进入路由实例
	MongoDB shell version: 3.2.1
	connecting to: test
	Server has startup warnings:
	2018-09-15T16:00:19.746+0800 I CONTROL  [main] ** WARNING: You are running this process as the root user, which is not recommended.
	2018-09-15T16:00:19.746+0800 I CONTROL  [main]
	mongos> sh.status()
	--- Sharding Status ---
	  sharding version: {
		"_id" : 1,
		"minCompatibleVersion" : 5,
		"currentVersion" : 6,
		"clusterId" : ObjectId("5b9cbc14b4c77895df796bac")
	}
	  shards:                        #shards下为空，没有分片服务器
	  active mongoses:
		"3.2.1" : 1
	  balancer:
		Currently enabled:  yes
		Currently running:  no
		Failed balancer rounds in last 5 attempts:  0
		Migration Results for the last 24 hours:
			No recent migrations
	  databases:

# 通过sh.addShard命令添加两个分片服务器，

	mongos> sh.addShard("192.168.3.102:27017")
	{ "shardAdded" : "shard0000", "ok" : 1 }
	mongos> sh.addShard("192.168.3.103:27017")
	{ "shardAdded" : "shard0001", "ok" : 1 }

# 再次查看分片信息，可以看到shards：选项下已经显示刚刚添加的分片服务器
	mongos> sh.status()
	--- Sharding Status --- 
	  sharding version: {
			"_id" : 1,
			"minCompatibleVersion" : 5,
			"currentVersion" : 6,
			"clusterId" : ObjectId("6123c015cdd4b633d3a86325")
	  }
	  shards:
			{  "_id" : "shard0000",  "host" : "192.168.3.102:27017" }
			{  "_id" : "shard0001",  "host" : "192.168.3.103:27017" }
	  active mongoses:
			"3.2.22" : 1
	  balancer:
			Currently enabled:  yes
			Currently running:  no
			Failed balancer rounds in last 5 attempts:  0
			Migration Results for the last 24 hours: 
					1689 : Success
					2 : Failed with error 'aborted', from shard0001 to shard0000
	  databases:
			{  "_id" : "school",  "primary" : "shard0001",  "partitioned" : true }
					school.info
							shard key: { "id" : 1 }
							unique: false
							balancing: true
							chunks:
									shard0000       1689
									shard0001       3312
							too many chunks to print, use verbose if you want to force print
			{  "_id" : "test",  "primary" : "shard0000",  "partitioned" : false }

	mongos>

第六步：实现分片功能

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
#添加两个分片服务器后，数据库和集合还未启用分片
	mongos> show dbs
	config  0.031GB
	mongos> use school          #进入并创建数据库school
	switched to db school
	mongos> for (var i=1;i<=50000;i++)db.info.insert({"id":i,"name":"tom"+i})           #创建集合info，并使用循环插入50000条数据

	WriteResult({ "nInserted" : 1 })

	#使用sh.enableSharding("school")命令启用school数据库分片
	mongos> sh.enableSharding("school")           
	{ "ok" : 1 }

	#针对info集合创建索引
	mongos> db.info.createIndex({"id":1})
	{
		"raw" : {
			"192.168.30.55:47017" : {
				"createdCollectionAutomatically" : false,
				"numIndexesBefore" : 1,
				"numIndexesAfter" : 2,
				"ok" : 1
			}
		},
		"ok" : 1
	}

	#使用sh.shardCollection("school.info",{"id":1})命令对集合info进行分片

	mongos> sh.shardCollection("school.info",{"id":1})
	{ "collectionsharded" : "school.info", "ok" : 1 }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
	
for (var i=50000;i<=1000000;i++)db.info.insert({"id":i,"name":"tom"+i})

根据 id 自动分片到 shard1 ，shard2上面去。要这样设置是因为不是所有mongodb 的数据库和表都需要分片！

#查看分片情况如下，此处省略其他信息
db.集合.stats();



#查看mongo集群是否开启了 balance 状态
mongos> sh.getBalancerState()
true
1
2


#也可以通过在路由节点mongos上执行sh.status() 查看balance状态。如果balance开启，查看是否正在有数据的迁移
mongos> sh.isBalancerRunning()
false
1
2


#如果未开启，执行命令
sh.setBalancerState( true )




#设置分片chunk大小
use config
db.settings.save({ "_id" : "chunksize", "value" : 1 })  #设置1M是为了测试，否则要插入大量数据才能分片。

其他操作

查看分片信息

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
mongos> sh.status()
--- Sharding Status ---
  sharding version: {
    "_id" : 1,
    "minCompatibleVersion" : 5,
    "currentVersion" : 6,
    "clusterId" : ObjectId("5b9cbc14b4c77895df796bac")
}
  shards:                             
    {  "_id" : "shard0000",  "host" : "192.168.30.55:47017" }              #两个分片服务器信息
    {  "_id" : "shard0001",  "host" : "192.168.30.55:47018" }
  active mongoses:
    "3.2.1" : 1
  balancer:
    Currently enabled:  yes
    Currently running:  no
    Failed balancer rounds in last 5 attempts:  0
    Migration Results for the last 24 hours:
        5 : Success
  databases:
    {  "_id" : "school",  "primary" : "shard0000",  "partitioned" : true }             #数据库school的分片信息
        school.info
            shard key: { "id" : 1 }
            unique: false
            balancing: true
            chunks:                               #可以看到chunks均匀分布到两个分片上
                shard0000    6
                shard0001    5
            { "id" : { "$minKey" : 1 } } -->> { "id" : 4682 } on : shard0001 Timestamp(2, 0)
            { "id" : 4682 } -->> { "id" : 9364 } on : shard0001 Timestamp(3, 0)
            { "id" : 9364 } -->> { "id" : 14046 } on : shard0001 Timestamp(4, 0)
            { "id" : 14046 } -->> { "id" : 18728 } on : shard0001 Timestamp(5, 0)
            { "id" : 18728 } -->> { "id" : 23410 } on : shard0001 Timestamp(6, 0)
            { "id" : 23410 } -->> { "id" : 28092 } on : shard0000 Timestamp(6, 1)
            { "id" : 28092 } -->> { "id" : 32774 } on : shard0000 Timestamp(1, 6)
            { "id" : 32774 } -->> { "id" : 37456 } on : shard0000 Timestamp(1, 7)
            { "id" : 37456 } -->> { "id" : 42138 } on : shard0000 Timestamp(1, 8)
            { "id" : 42138 } -->> { "id" : 46820 } on : shard0000 Timestamp(1, 9)
            { "id" : 46820 } -->> { "id" : { "$maxKey" : 1 } } on : shard0000 Timestamp(1, 10)

添加标签

1
2
3
4
5
6
7
8
9
10
11
12
13
14
mongos> sh.addShardTag("shard0000","abc01")
mongos> sh.addShardTag("shard0001","abc02")

mongos> sh.status()
--- Sharding Status ---
  sharding version: {
    "_id" : 1,
    "minCompatibleVersion" : 5,
    "currentVersion" : 6,
    "clusterId" : ObjectId("5b9cbc14b4c77895df796bac")
}
  shards:
    {  "_id" : "shard0000",  "host" : "192.168.30.55:47017",  "tags" : [ "abc01" ] }
    {  "_id" : "shard0001",  "host" : "192.168.30.55:47018",  "tags" : [ "abc02" ] }

MongoDB分片服务器管理

根据需求可以添加或删除sharding server。

1
2
3
4
5
6
7
8
9
10
11
[root@localhost bin]# cp mongodb3.conf mongodb4.conf
[root@localhost bin]# vim mongodb4.conf

port=47019
logpath=/data/logs/mongodb4.log
dbpath=/data/mongodb4
fork=true
logappend=true
maxConns=5000
storageEngine=mmapv1
shardsvr=true

启动实例4，进入路由实例添加新的分片服务器

1
2
3
4
5
6
7
8
9
10
[root@localhost bin]# mongod -f mongodb4.conf
about to fork child process, waiting until server is ready for connections.
forked process: 52634
child process started successfully, parent exiting
[root@localhost bin]# mongo
MongoDB shell version: 3.2.1
connecting to: test
Server has startup warnings:
2018-09-15T16:00:19.746+0800 I CONTROL  [main] ** WARNING: You are running this process as the root user, which is not recommended.
2018-09-15T16:00:19.746+0800 I CONTROL  [main]

使用sh.addShard(“192.168.30.55:47019”) 命令添加一个新的分片服务器，再次查看分片信息，发现分片均匀地分布到3个分片服务器上

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
mongos> sh.addShard("192.168.30.55:47019")          #添加分片服务器
{ "shardAdded" : "shard0002", "ok" : 1 }
mongos> sh.status()                   #查看分片服务信息
--- Sharding Status ---
  sharding version: {
    "_id" : 1,
    "minCompatibleVersion" : 5,
    "currentVersion" : 6,
    "clusterId" : ObjectId("5b9cbc14b4c77895df796bac")
}
  shards:
    {  "_id" : "shard0000",  "host" : "192.168.30.55:47017" }
    {  "_id" : "shard0001",  "host" : "192.168.30.55:47018" }
   {  "_id" : "shard0002",  "host" : "192.168.30.55:47019" }
  active mongoses:
    "3.2.1" : 1
  balancer:
    Currently enabled:  yes
    Currently running:  no
    Failed balancer rounds in last 5 attempts:  0
    Migration Results for the last 24 hours:
        8 : Success
  databases:
    {  "_id" : "school",  "primary" : "shard0000",  "partitioned" : true }
        school.info
            shard key: { "id" : 1 }
            unique: false
            balancing: true
            chunks:
                shard0000    4
                shard0001    4
                shard0002    3
            { "id" : { "$minKey" : 1 } } -->> { "id" : 4682 } on : shard0002 Timestamp(9, 0)
            { "id" : 4682 } -->> { "id" : 9364 } on : shard0001 Timestamp(9, 1)
            { "id" : 9364 } -->> { "id" : 14046 } on : shard0001 Timestamp(4, 0)
            { "id" : 14046 } -->> { "id" : 18728 } on : shard0001 Timestamp(5, 0)
            { "id" : 18728 } -->> { "id" : 23410 } on : shard0001 Timestamp(6, 0)
            { "id" : 23410 } -->> { "id" : 28092 } on : shard0002 Timestamp(7, 0)
            { "id" : 28092 } -->> { "id" : 32774 } on : shard0002 Timestamp(8, 0)
            { "id" : 32774 } -->> { "id" : 37456 } on : shard0000 Timestamp(8, 1)
            { "id" : 37456 } -->> { "id" : 42138 } on : shard0000 Timestamp(1, 8)
            { "id" : 42138 } -->> { "id" : 46820 } on : shard0000 Timestamp(1, 9)
            { "id" : 46820 } -->> { "id" : { "$maxKey" : 1 } } on : shard0000 Timestamp(1, 10)

使用db.runCommand({“removeshard”:”192.168.30.55:47019”})命令可以删除新添加的分片服务器

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
mongos> use admin           # 注：在admin db下执行命令。
switched to db admin
mongos> db.runCommand({"removeshard":"192.168.30.55:47019"})
{
    "msg" : "draining started successfully",
    "state" : "started",
    "shard" : "shard0002",
    "note" : "you need to drop or movePrimary these databases",
    "dbsToMove" : [ ],
    "ok" : 1
}
mongos> db.runCommand({"removeshard":"192.168.30.55:47019"})
{
    "msg" : "removeshard completed successfully",
    "state" : "completed",
    "shard" : "shard0002",
    "ok" : 1
}

#“注意：该命令至少执行两次才能成功删除，执行到state为completed才真正删除，否则就是没有删除成功，该分片处于"draining" : true状态，该状态下不但该分片没有删除成功，而且还影响接下来删除其他分片操作，遇到该状态再执行一次removeshard即可，最好就是删除分片时一直重复执行删除命令，直到state为completed再次查看分片信息，可以看到分片已删除

mongos> sh.status()
--- Sharding Status ---
  sharding version: {
    "_id" : 1,
    "minCompatibleVersion" : 5,
    "currentVersion" : 6,
    "clusterId" : ObjectId("5b9cbc14b4c77895df796bac")
}
  shards:
    {  "_id" : "shard0000",  "host" : "192.168.30.55:47017" }
    {  "_id" : "shard0001",  "host" : "192.168.30.55:47018" }
  active mongoses:
    "3.2.1" : 1
  balancer:
    Currently enabled:  yes
    Currently running:  no
    Failed balancer rounds in last 5 attempts:  0
    Migration Results for the last 24 hours:
        11 : Success
  databases:
    {  "_id" : "school",  "primary" : "shard0000",  "partitioned" : true }
        school.info
            shard key: { "id" : 1 }
            unique: false
            balancing: true
            chunks:
                shard0000    6
                shard0001    5
            { "id" : { "$minKey" : 1 } } -->> { "id" : 4682 } on : shard0000 Timestamp(10, 0)
            { "id" : 4682 } -->> { "id" : 9364 } on : shard0001 Timestamp(9, 1)
            { "id" : 9364 } -->> { "id" : 14046 } on : shard0001 Timestamp(4, 0)
            { "id" : 14046 } -->> { "id" : 18728 } on : shard0001 Timestamp(5, 0)
            { "id" : 18728 } -->> { "id" : 23410 } on : shard0001 Timestamp(6, 0)
            { "id" : 23410 } -->> { "id" : 28092 } on : shard0001 Timestamp(11, 0)
            { "id" : 28092 } -->> { "id" : 32774 } on : shard0000 Timestamp(12, 0)
            { "id" : 32774 } -->> { "id" : 37456 } on : shard0000 Timestamp(8, 1)
            { "id" : 37456 } -->> { "id" : 42138 } on : shard0000 Timestamp(1, 8)
            { "id" : 42138 } -->> { "id" : 46820 } on : shard0000 Timestamp(1, 9)
            { "id" : 46820 } -->> { "id" : { "$maxKey" : 1 } } on : shard0000 Timestamp(1, 10)

连接配置服务器查看信息

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
#配置服务器存储了MongoDB数据库集合分片的详细信息，可以通过以下命令查看
mongo --port 37017

configsvr> use config

configsvr> show collections  

....
collections
chunks
databases   
....

configsvr> db.chunks.findOne()

configsvr> db.collections.find()

configsvr> db.databases.find()
  • 版权声明： 本博客所有文章除特别声明外，著作权归作者所有。转载请注明出处！
  • Copyrights © 2019-2023 XIN LONG
  • 访问人数: | 浏览次数:

请我喝杯咖啡吧~

支付宝
微信