Monitoring can detect problems before end users notice them, help plan for growth and make troubleshooting easier when an unexpected problem does occur.
Use systems like Ganglia and Cacti to gather statistics from all servers and network devices. Building up graphs of these statistics allows you to see normal usage patterns and spot any spikes more quickly. These graphs also allow you to visualize what conditions were like during an incident that occurred temporarily.
Ensure log messages are being aggregated by Syslog. Using a tool like LogAnalyzer to visualize the log messages in colour makes it much easier to spot unusual activity.
Use an alerting tool such as Nagios or Icinga to monitor the graphs (see Ganglia-Nagios-Bridge) and logs (see Syslog-Nagios-Bridge)and also monitor the availability of services such as the SIP and TURN processes and the expiry dates of TLS certificates.
Appendix C, Configuring Nagios to monitor SIP, XMPP and TURN contains specific instructions to configure Nagios/Icinga to monitor SIP, XMPP and TURN servers.
Check /var/log/syslog and
/var/log/daemon.log.
Many of the RTC processes can also create their own logfiles in other locations. Refer to the configuration files to see which type of logging each process uses.
The repro daemon web interface has a control
on the Settings page where the log level
can be changed at runtime without restarting the daemon.
For those processes that have a web interface, this can be a useful way to see runtime activity at a glance.
The repro daemon web interface has a
Registrations page where all known SIP
registrations can be seen.
Example 5.2, “Inspecting DNS entries with dig” demonstrates how to use the
dig utility to verify that the DNS entries exist.
Use the netstat or lsof commands to
verify that each process is listening on the correct IP addresses
and ports.
If a process is failing to start and the reason is not
clear in the log file or console output, use the strace
utility to determine whether any syscall
is failing.
the section called “Testing with s_client” demonstrates how
to use the OpenSSL s_client
utility to make a test connection to the SIP proxy on the TLS and
WebSocket over TLS ports.
Use tcpdump to capture the packets to a file
on the server. Copy the capture file to a workstation and inspect
it with Wireshark.
For SIP over UDP or TCP, without encryption, the
ngrep utility can be useful for identifying packets
containing a particular string.
Run the process in debug mode, in the foreground on a terminal,
to see what it is doing. Running the repro daemon
this way will allow you to see the SIP messages in the console.