collectd

Introduction

Collecting and visualizing data is an important way to make informed decisions about your servers and projects.

In a previous guide, we discussed how to install and configure Graphite to visualize data on our servers. However, we didn’t have a good way of collecting or even passing data into Graphite.

In this guide, we’ll discuss the installation and use of collectd, a system statistics gatherer that can collect and organize metrics about your server and running services.

We will show you how to install and configure collectd to pass data into Graphite to render. We will assume that you have Graphite up and running on an Ubuntu 14.04 server as we showed you in the last guide.

Deploy Graphite with Grafana on Ubuntu 14.04(Part Ⅰ)

Graphite是一款企业级的监控工具,其采用Django框架,可以运行在廉价的硬件资源之上。Graphite采用简单的文本协议和绘图功能可以方便地使用在任何操作系统上。本文将以Ubuntu 14.04为系统环境,就Graphite与Grafana的安装与协作做详细介绍。

Before You Begin / 热身准备

1. Familiarize yourself with our Getting Started guide and complete the steps for setting your Linode’s hostname and timezone.
本文原文来自Linode官网,所以会看到Linode字样,不用理会,把它当成你的主机的就可以了

Python日期自然月加减实操

这段代码用于对数据库中两个表中不同形式的时间字段进行月份变更操作,月份的变更以自然月形式形式进行。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
#!/usr/bin/python
# By Gavin
# Date: 20190909
# Decription: 实现日期的自然月相加减,比如8月31号加一个月是9月30,而不是9月31号或10月1号
import time
import MySQLdb
import math,datetime

def _add_month_interval(dt,inter):
m=dt.month+inter-1
y=dt.year
y=dt.year+int(m/12)
m=m % 12 +1
return (y,m)

def add_month_interval(dt,inter):
y,m=_add_month_interval(dt,inter)
y2,m2=_add_month_interval(dt,inter+1)
maxD=( datetime.date(y2,m2,1)-datetime.timedelta(days=1) ).day
d= dt.day<=maxD and dt.day or maxD
#return datetime.date(y,m,d)
return(y,m,d)

def add_year_interval(dt,inter):
return add_month_interval(dt,inter*12)
# 上面三个函数与业务无关,可直接嫁接使用

# inter为需要变更的月数,需要加一个月就写1,减两个月就写-2
def update_metric(inter):
conn=MySQLdb.connect(host='localhost',user='root',passwd='root',db='gavin',port=3306)
cur=conn.cursor()
cur.execute('select date from metric_impressions order by date desc')
#此时获取到的结果为元组
results = cur.fetchall()
#将元组转换为列表
result=list(results)
for now_date in result:
# now_date[0]格式形如:2016-09-21 12:03:23
timeArray = time.strptime(str(now_date[0]), "%Y-%m-%d %H:%M:%S")
# time.strptime处理过的数据结构(元组)不能被重新赋值
# 所以这里需要将其转化为列表
timeArray = list(timeArray)
dt = datetime.date(timeArray[0],timeArray[1],timeArray[2])
timeArray[0],timeArray[1],timeArray[2] = add_month_interval(dt,inter)
new_date = time.strftime("%Y-%m-%d %H:%M:%S", timeArray)
print "new_date:{} now_date:{}".format(new_date, now_date[0])
sql = 'update metric_impressions_hour set date="{}" where date="{}"'.format(new_date, now_date[0])
cur.execute(sql)
conn.commit()
conn.close()

def update_compaign(inter):
conn=MySQLdb.connect(host='localhost',user='root',passwd='root',db='gavin',port=3306)
cur=conn.cursor()
cur.execute('select data_date from campaign order by data_date desc')
results = cur.fetchall()
result=list(results)
for now_date in result:
# now_date[0]格式形如:20160921
timeArray = time.strptime(str(now_date[0]), "%Y%m%d")
timeArray = list(timeArray)
dt = datetime.date(timeArray[0],timeArray[1],timeArray[2])
timeArray[0],timeArray[1],timeArray[2] = add_month_interval(dt,inter)
new_date = time.strftime("%Y%m%d", timeArray)
print "new_date:{} now_date:{}".format(new_date, now_date[0])
sql = 'update campaign_stats set data_date="{}" where data_date="{}"'.format(new_date, now_date[0])
print sql
cur.execute(sql)
conn.commit()
conn.close()

update_compaign(1)

Percona容器部署

镜像编译

以镜像percona为例,假如我们应用场景如下:
现在需要向pernoa镜像中添加数据库自动备份功能

下载镜像

1
2
cd ~
git clone ssh://gavinwei@git.example.com:29418/docker/bd/percona

操作percona代码

1
2
3
# cd ~/percona
# ls
pom.xml proguard.conf src

我们所需要操作都在src目录的最底层:

SS5 Installation

安装ss5

apt-get install gcc make

下载源码包

1
2
3
wget http://120.52.73.44/jaist.dl.sourceforge.net/project/ss5/ss5/3.8.9-8/ss5-3.8.9-8.tar.gz
cd ./ss5-3.8.9-8
./configure

报错:
configure: error: *** Some of the headers weren't found ***
安装libpam0g-dev解决:
apt-get install libpam0g-dev

make报错:

1
2
3
4
5
SS5OpenLdap.c:29:18: fatal error: ldap.h: No such file or directory
#include <ldap.h>
^
compilation terminated.
make[1]: *** [SS5OpenLdap.o] Error 1

解决:
apt-get install libcurl4-openssl-dev