optimize gs_dbmind and add docs

Offering: GaussDB Kernel

More detail: opt
This commit is contained in:
w00464751 2022-03-26 17:45:02 +08:00 committed by wangtq
parent 2d8130e1b5
commit 5fbf995e8f
4 changed files with 173 additions and 0 deletions

View File

@ -0,0 +1,136 @@
# DBMind
DBMind is a part of openGauss, which empowers openGauss to carry the autonomous operations and maintenance capabilities. DBMind is leading and open-source. Through DBMind, users can easily discover database problems and the root causes of the problems in seconds.
![DBMind overview](docs/dbmind.png)
# Getting Started
### Prerequisites
In order to run DBMind, the following components should be configured and running.
#### Python Runtime
At least Python 3.6.
#### Third-party Dependencies
Use `pip install` to install the python dependencies.
Type the `pip install` command with dependencies according to the environment you are running:
```
pip install -r requirements-aarch64.txt | requirements-x86.txt
```
#### Prometheus up and Running
Download and run the [Prometheus](https://prometheus.io/) time-series database.
#### Node Exporter
Download and run the [Prometheus node exporter](https://prometheus.io/download/#node_exporter). Node-exporter is to monitor the Linux system. Hence, one Linux environment only needs to deploy one node-exporter.
### DBMind Components
The following DBMind components are required:
**Note: If you want to get higher security, you should use the HTTPS scheme.**
#### openGauss Exporter
The openGauss-exporter reads data from the database and places it on the Prometheus time-series database.
OpenGauss-exporter is to monitor only one database instance. So if your deployment environment has not only one instance, you should start multiple openGauss-exporters to correspond to monitor multiple database instances.
It needs database access with a user having the role of at least **monadmin** (monitoring administrator) granted to run it. For example, you can grant monadmin privilege to role dbmind as below:
```
ALTER USER dbmind monadmin;
```
Use the following command with the parameters below:
```
gs_dbmind component opengauss_exporter ...
```
You can get detailed explanations of this component through passing `--help`:
```
gs_dbmind component opengauss_exporter --help
```
For example, the following command starts it:
```
gs_dbmind component opengauss_exporter --url postgresql://username:password@host:port/database --web.listen-address 0.0.0.0 --web.listen-port 9187 --log.level warn --disable-https ...
```
To test that the exporter is up, type the following command on its host (or use change the localhost to the server address):
```
curl -vv http://localhost:9187/metrics
```
#### Reprocessing Exporter
Reprocessing-exporter is a re-processing module for metrics stored in the Prometheus server. It helps Prometheus to reprocess the metric data then dump the new data into Prometheus. Therefore, only one needs to be started in a deployment environment.
To run it use the command below:
```
gs_dbmind component reprocessing_exporter ...
```
Users can see usage by using `--help` too.
See this example for running the exporter in a single machine development environment:
```
gs_dbmind component reprocessing_exporter 127.0.0.1 9090 --web.listen-address 0.0.0.0 --web.listen-port 9189
```
Use the following command to check that the service is up:
```
curl http://127.0.0.1:9189/metrics
```
### Configure, Start and Stop the DBMind Service
DBMind service is a memory-resident backend service. Therefore, users should configure it first then start or stop the service by using the configuration.
Service usages:
$ gs_dbmind service --help
usage: service [-h] -c DIRECTORY [--only-run {...}] [--interactive | --initialize] {setup,start,stop}
positional arguments:
{setup,start,stop} perform an action for service
optional arguments:
-h, --help show this help message and exit
-c DIRECTORY, --conf DIRECTORY
set the directory of configuration files
--only-run {slow_query_diagnosis,forecast}
explicitly set a certain task running in the backend
--interactive configure and initialize with interactive mode
--initialize initialize and check configurations after configuring.
#### Configure
DBMind offers two methods to configure. The one is an interactive mode by using `--interactive` argument, the other is a modification by hands.
See this example for configuring in the interactive mode:
```
gs_dbmind service setup -c CONF_DIRECTORY --interactive
```
Then users can type parameters into the shell terminal.
See the following example for configuring by hands:
```
gs_dbmind service setup -c CONF_DIRECTORY
```
After executing the above command, the directory `CONF_DIRECTORY` will generate too many configuration files. Therefore, users should modify these parameters in the `CONF_DIRECTORY/dbmind.conf`. While users finish configuring, this command needs to be run to initialize DBMind according to the `CONF_DIRECTORY/dbmind.conf`.
```
gs_dbmind service setup -c CONF_DIRECTORY --initialize
```
#### Start or Stop the DBMind Service
After configuring, specify your CONF_DIRECTORY, users can start or stop the service directly.
```
gs_dbmind service start/stop -c CONF_DIRECTORY
```
### Component
If users want to use a specific component offline. They can use the sub-command `component`:
```
gs_dbmind component xxx ...
```
`xxx` is the name of a component. Users can also get the component list by using the `--help` argument.
For example, use the following component to tune the knobs of a database:
```
gs_dbmind component xtuner --help
```
# LICENSE
Mulan PSL v2

Binary file not shown.

After

Width:  |  Height:  |  Size: 108 KiB

View File

@ -0,0 +1,30 @@
# DBMind Glossary
*If you add new entries, keep the alphabetical sorting!*
## Glossary
``agent``
Firstly, DBMind diagnoses the database instance by using monitoring metrics, then stores the diagnosis results. But if users want DBMind to perform some operations which can heal the database instance, under this scenario, DBMind needs an agent to receive command from DBMind then perform it. This is the responsibility of the agent.
``alarm``
When DBMind detects abnormal events, the notification generated is called alarm.
Alarm sources include many aspects, such as from historical logs, anomalies from historical KPI, and forecasts of KPI.
``anomaly detection``
Anomaly Detection (AD, aka outlier analysis) is a step in data mining that identifies events, and/or observations that deviate from a dataset's normal behavior. In the DBMind, we use it to detect abnormal events from monitoring metrics.
``component``
Plugins of DBMind. The components are all independent functionalities. Users can execute them by using shell command `gs_dbmind component`, also can call/import them from DBMind.
``confile``
The name of configuration file.
``confpath``
The directory path which stores common data, including configuration, log, pid, etc.
``exporter``
[Prometheus](https://prometheus.io/)'s plugin. The purpose of exporters is to scrape metrics from monitored object. See more via [Prometheus exporter](https://prometheus.io/docs/instrumenting/exporters/).
``metadatabase``
The database which you configure in the configuration file is to store monitoring, prediction and forecast result in periodical tasks. Hence, the response time for users to browse the results is quicker.

View File

@ -38,6 +38,13 @@ function extract_libname() {
ABSPATH="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
DBMIND_PATH=${ABSPATH}/dbmind
if [[ ! -d $DBMIND_PATH && -d "$ABSPATH/tools" ]]; then
die "Not found the subdirectory 'dbmind' but found 'tools'. Maybe you are in the source code directory.\n
If you want to debug DBMind in this directory, you should rename the name of subdirectory 'tools' to 'dbmind', as follows: \n
> mv tools dbmind
"
fi
# Check whether Python has been installed onto the OS.
PYTHON=$(which python)
if [[ "$?" != "0" ]]; then