CLI reference guide
Introduction
The utility hamonitor is used to perform all monitor actions. It is supplied as part of the docker image. To run the utility first gain shell access to the running docker instance:
# docker exec -it hamonitor bash
The hamonitor utility is self-documented; typing any command or subcommand with no arguments produces a help summary:
# hamonitor
NAME:
hamonitor - RSF-1 shared resources monitor
USAGE:
hamonitor [global options] command [command options] [arguments...]
VERSION:
2.1
COMMANDS:
alert Alert management
iscsi iSCSI initiator
login Log in to RSF-1 shared resources monitor
logout Sign out of RSF-1 shared resources monitor
resource Resource management
user User management
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--help, -h show help
--version, -v print the version
User management
Administration of users is done using the user subcommand. It allows
for user creation, deletion and listing of existing users (modification
is not supported in this release).
User addition
To create a new user, enter the subcommand:
# hamonitor user create
Enter username: oper
Enter password: *********
Verify password: *********
Enter real name [None]: Operator
Enter email address [None]: ops@example.com
Available roles: 0 (view only), 1 (operator), 2 (admin)
Enter role [0]: 1
User oper successfully created
Listing users
To show a list of configured users enter the subcommand:
# hamonitor user list
(JSON output...)
Note that the resulting list is provided in JavaScript Object Notation
(JSON) format; to make it more readable pipe the output through jq
(a utility to print JSON into a more human readable format, shipped
with the docker image). Piping the above command through
jq results in:
# hamonitor user list | jg
[
{
"userid": 1,
"creation_time": "1600692753.915523",
"username": "admin",
"realname": "",
"password": "<*** HIDDEN ***>",
"role": 2,
"enabled": "True"
},
{
"userid": 2,
"creation_time": "1600701250.404133",
"username": "user",
"realname": "",
"password": "<*** HIDDEN ***>",
"role": 1,
"enabled": "True"
}
]
Deleting a user
To delete a user enter the subcommand:
# hamonitor user delete
Enter userid: 2
Do you really want to remove user 2? [y/n]: y
User successfully deleted
Note that users are referenced by their userid -
these can be shown using the user list subcommand.
Logging in and out
Before any changes are made to the monitor, it is necessary to authenticate using the login subcommand. The connection URL uses localhost and port 13514 by default - an alternative can be entered when logging in:
# hamonitor login
Enter URL [https://localhost:13514 if empty]:
Enter Username: admin
Enter Password:
Welcome admin
# hamonitor logout
Resource management
Introduction
Resources are managed using the hamonitor resource subcommand. Note
that the monitor does not create the monitored resource and does not
modify the access rights of a monitored resource, rather it monitors
the availability of an existing resource.
Adding a monitored resource
To configure a monitored resource, use the resource add subcommand:
# hamonitor resource add
Enter protocol [NFS, SMB, iSCSI, LDAP]: SMB
IP address of the SMB server: 192.168.4.1
SMB share name: Scratch
Mount options [return for None]:
User name [return for None]: system
Password [return for None]:
Do you want to test the connection now? [y/n]y
Test successful.
201: Created
Authentication for a monitored resource
When adding a monitored resource, the username and password fields are not required when the resource is publicly shared, i.e. where no authentication is required to access the resource.
In the case where authentication is required to access a resource, then credentials should be supplied using the username and password fields during addition of the monitored resource.
Removing a monitored resource
To remove a resource from the monitor use the resource remove
subcommand along with the resource ID to identify the resource to
be removed:
# hamonitor resource remove --id 1
200: OK
Enabling and Disabling a monitored resource
The active monitoring state of a resource can be toggled between enabled or disabled. When a resource is first added, its monitor state is enabled. To suspend monitoring without removing the resource entirely (say for example the resource is going offline for maintenance) its state can be set to disabled in the monitor. Reinstate monitoring for a resource by enabling it.
To disable a resource use the resource disable subcommand:
# hamonitor resource disable --id 1
200: OK
resource enable subcommand:
# hamonitor resource enable --id 1
200: OK
Listing resources being monitored
To list all resources configured use the resource list subcommand:
# hamonitor resource list
...
jq for a more human readable form.
Status of an individual resource
To check the status of individual resources use the resource status subcommand:
# hamonitor resource status --id 1 | jq
{
"path": "Scratch",
"ip": “192.168.4.1”,
"protocol": "SMB",
"enabled": true,
"creationDate": "2020-09-22T12:16:54+00:00",
"notifications": {
"slack": false,
"teams": false,
"email": false,
"snmp": false
}
}
iSCSI resource monitoring
Introduction
As of release 2.1 the ability to perform iSCSI resource monitoring has been added. The monitor acts as an initiator and is connected to the remote target to be monitored.
For Linux hosts to take advantage of this feature it is necessary to
map the hosts /lib/modules directory to the container using the docker
--volume option when the container is started:
# docker ...\
--volume /lib/modules:/lib/modules\
...
iscsi_tcp) can be loaded by the hamonitor running in the
container, a process which in turn triggers docker to load the module
in the docker host, thereby providing iSCSI kernel features to the
process running in the docker container - this is a feature of docker.
Note the iSCSI module is only loaded in the docker host, it does not run in the container. The process of running the iSCSI module in the host exposes its facilities to clients in the docker container, thereby enabling iSCSI monitoring. If you do not intend to run any iSCSI monitoring, it is not necessary to map the modules library
The iSCSI initiator
To monitor an iSCSI resource, it is first necessary to define a local
initiator within the scope of the monitor so connections can be made
to remote targets. By default, the monitor generates a suitable
initiator to use in iSCSI connections. To view and manipulate the
default iSCSI initiator the hamonitor iscsi sub-command is used:
# hamonitor iscsi
NAME:
hamonitor iscsi – iSCSI initiator
USAGE:
hamonitor iscsi command [command options] [arguments…]
COMMANDS:
modify, m Modify system’s iSCSI initiator name
restart, r. Restart open-iscsi daemon
show, s. Show current iSCSI initiator name
OPTIONS:
--help, -h show help
To display the current iSCSI initiator name:
# hamonitor iscsi show
iqn.1993-08.org.debian:01:c9159016459f
Changing the initiator name
It may be necessary to change the default initiator name, for example when more than one instance of the monitor is being run on the same host or where iSCSI authentication is based on the initiator name1 and a specific format/name is required. The initiator name is changed using the iscsi modify command:
# hamonitor iscsi modify
Enter the new iSCSI initiator name: iqn.1993-08.org.debian:01:c91590164000
New iSCSI initiator name set to iqn.1993-08.org.debian:01:c91590164000
# hamonitor iscsi restart
iSCSI module restarted
iscsi show command:
# hamonitor iscsi show
iqn.1993-08.org.debian:01:c91590164000
Monitoring an iSCSI resource
In order to monitor an iSCSI target it is added as a resource to the hamonitor. The following example adds monitoring for an iSCSI target using CHAP authentication2:
# hamonitor resource add
Enter protocol [NFS, SMB, iSCSI, LDAP]: iSCSI
IP:PORT of the iSCSI Target Portal: 192.168.5.2:3263
Target IQN: iqn.2021-05.com.vmlab-nef:target0
Mount options [return for None]:
Username [return for None]: hac
Password [return for None]: <hidden>
Do you want to test the connection now? [y/n] y
Test successful
201: Created
Use the resource list command to display the newly added resource:
# hamonitor resource list | jq
{
"path": "iqn.2021-05.com.vmlab-nef:target0",
"ip": "192.168.5.2:3263",
"protocol": "iSCSI",
"enabled": true,
"creationDate": "2021-05-19T19:18:00+00:00",
"notifications": {
"slack": false,
"teams": false,
"email": false,
"snmp": false
},
"activeIssues": false,
"status": "online",
"onlineSince": "2021-05-19T19:18:09+00:00",
"uptime": 78,
"issues": {
"total": 1,
"lastHour": 1,
"lastSixHours": 1,
"last24Hours": 1,
"lastWeek": 1,
"totalDowntime": 6,
"averageDowntimePerIssue": 6,
"lastIssueOpen": "2021-05-19T19:18:03+00:00",
"lastIssueClose": "2021-05-19T19:18:09+00:00"
},
"availability": 0.9323,
"resourceid": 15
}
# hamonitor alert teams create -id 15
Enter Teams hook URL: https://highavailabilitycom.webhook.office.com/webhookb2/...
LDAP resource monitoring
Introduction
As of release 2.1 the ability to monitor LDAP servers has been added. Monitoring of LDAP servers is autonomous and does not require login information on the remote server.
Monitoring an LDAP resource
In order to monitor an LDAP server it is added as a resource to the hamonitor:
# hamonitor resource add
Enter protocol [NFS, SMB, iSCSI, LDAP]: LDAP
IP address of the LDAP server: 10.5.4.33
Distinguished name (DN): ou=group,dc=mydomain,dc=com
Do you want to test the connection now? [y/n]y
Test successful
201: Created
# hamonitor resource list | jq
{
"path": "ou=group,dc=mydomain,dc=com",
"ip": "10.5.4.33",
"protocol": "LDAP",
"enabled": true,
"creationDate": "2021-05-19T19:18:00+00:00",
"notifications": {
"slack": false,
"teams": false,
"email": false,
"snmp": false
},
"activeIssues": false,
"status": "online",
"onlineSince": "2021-05-19T19:18:09+00:00",
"uptime": 78,
"issues": {
"total": 1,
"lastHour": 1,
"lastSixHours": 1,
"last24Hours": 1,
"lastWeek": 1,
"totalDowntime": 6,
"averageDowntimePerIssue": 6,
"lastIssueOpen": "2021-05-19T19:18:03+00:00",
"lastIssueClose": "2021-05-19T19:18:09+00:00"
},
"availability": 0.9323,
"resourceid": 15
}
Alert management
The monitor supports several types of alerts. Alerts are configured on
a per-resource basis so each resource has its own alert
schema. Alerts are bound to resources using the mandatory --id
argument when adding an alert.
Adding an email alert
To configure an email alert, a valid SMTP server is required along with a user name and password. The monitor only supports email delivery over an encrypted TLS connection.
The (optional) TLS port on the email server is specified when the server address is entered:
```
# hamonitor alert email create --id 5
This alert method only supports authenticated email delivery over TLS.
Enter SMTP server address (MX:[PORT]): mx1.yourcomany.com:587
Enter SMTP server user name: realuseraccount@yourcompany.com
Enter SMTP server user password:
Verify password:
Enter email FROM address: alerts_sender_alias@yourcompany.com
Enter email TO address: alerts_manager_alias@yourcompany.com
201: Created
```
Adding a Slack alert
A slack alert will update a slack channel with events for which the resource has been configured. Before creating an alert, a slack webhook is required. Webhook's are created using the slack application itself, please see the slack documentation for how to create a suitable webhook.
Once a webhook link has been generated it is then used as part of the URL when creating the alert:
# hamonitor alert slack create --id 5
Enter Slack hook URL: https://hooks.slack.com/services/<link>
201: Created
Resource OFFLINE: nfs 192.168.22.6 /pool/nfs-share
Adding a Microsoft Teams alert
A teams alert will update a teams channel with events for which the resource has been configured. Before creating an alert a teams webhook is required. Webhook's are created from the teams application itself, please see the teams documentation for how to create a suitable webhook.
Once a webhook link has been generated it is then used as part of the URL when creating the alert:
# hamonitor alert teams create --id 5
Enter Teams hook URL: https://outlook.office.com/webhook/<link>
201: Created
Resource OFFLINE: smb 10.6.11.12 /pool/smb-share
Adding an SNMP alert / MIB location
To add an SNMP alert the IP address of an SNMP manager is added to the resource being monitored:
hamonitor alert snmp create --id 5
Enter SNMP manager address: 10.5.14.22
201: Created
/root/RSF-MIB.txt.
Changing the HTTPS authentication certificate
The CLI communicates with the monitor using its REST API over HTTPS
(TLS version 1.3). A self signed certificate is shipped in the docker
image as /root/cert.pem.
A site specific certificate can be used instead by installing it in
the shared host directory as cert.pem. It is necessary to restart the
docker container for it to pick up the new certificate; to restart the
container from the docker host run the command:
# docker restart hamonitor
Changing the frequency of the resource availability check
As delivered the monitor checks the availability of resources at a regular interval – this interval is known in the monitor as the frequency; its default value is 3 seconds.
In high latency or low speed networks, monitoring of resources could trigger false alarms due to higher-than-normal times the monitor needs to wait for responses. In cases such as these the monitoring frequency can be adjusted using the frequency subcommand.
To show the current frequency setting:
# hamonitor frequency show
# hamonitor frequency modify
Enter a new frequency in seconds (min 3s): 10
Frequency updated to 10 seconds