Help

Troubleshooting problems

First step in troubleshooting is to identify the cause of the problem you are facing. Once that is done, next on the list is gathering options on how to resolve the problem.

Determining the problem and its cause

When determining the problem, you can proceed according to the following algorithm. If the symptoms match with the problem you are experiencing, look below at the problem resolution.

Critical problems

Having a critical problem means that Dwarfguard is not providing the intended service - it is not able to track the status of managed devices. The reasons may be various, please proceed according to the algorithm outlined below.

  • Can you log into into the system?
    • NO: look into Accessing Dwarfguard below
    • YES: proceed to next question
  • Are you able to access the Dashboard page?
    • NO: you may try looking at the System -> Status and starting Dwarfguard service if possible or inspect event log in System -> Event log page but if that does not help, you will need to escalate this with your administrator or Support. You may also try restarting Dwarfguard service from command line if you manage your deployment yourself.
    • YES: proceed to next question
  • Does the Dashboard page show only a single light? (instead of 1 global and 4 breakdown lights)
    • YES: Dwarfguard service is not running. You may attempt to start it via System -> Status page but you would probably need to escalate it anyway to your administrator or Support.
    • NO: proceed to next question
  • Is the global status light (top menu bar, aka first dashboard light) red?
    • YES: Global status light in red color means that Dwarfguard service is not providing the intended service - the service itself is up but there is some blocker problem detected. Examples are exhausted or expired license. Inspect text accompanying the light and warning text below lights - it should shed some light on the source of the problem. You may also look into Licensing issues below.
    • NO: proceed to one of the following section:
      • Problems adding devices to Dwarfguard
      • Non-critical problems

Problems adding devices to Dwarfguard

If you are having problems connecting a new device to Dwarfguard, check that all of the following is okay. If you are unable to find any issue, you need to escalate the problem to support or your administrator.

  1. Dwarfguard service is running - the status light in the top bar is showing green or yellow light. If not, proceed according to Critical problems above.
  2. You are using correct agent type. If you try to use incompatible agent type, the process of adding the device would not work. Download the correct agent type.
  3. Your agent has inaccurate data. It boils down to:
    1. Correct URL. The URL in the server and on the Agent must be the same and it must be accessible from the device the agent is installed on. If your agent has a UI, check your URL and the result of ping and Dwarfguard healthcheck. If no UI is provided by the agent, try pinging the server from the device using command line.
    2. Correct SERVERID. The SERVERID is different for every server. Download the agent from the server and try again with the fresh download if server was reinstalled. Also, double-check that you are connecting to the correct server (see URL above).
    3. Valid server HTTPS certificate. If using a certificate signed by a third party, it is authenticated by the device own software (cURL, ca-certificates bundle). If using Dwarfguard self-signed certificate, it is part of the agent (do you have a fresh agent downloaded?). To make sure the problem is in the certificate, you may temporarily disable the certificate checking by setting the SECURITY agent's variable to 0 in /opt/adwarfg/settings.ini via command line. Also, if you do not use HTTPS but HTTP (e.g. on private secured trusted network), the certificate problem cannot happen.
    4. Required device software installed. The agent relies on having the cURL program installed and working. You may check that the cURL is there and working by calling it from a device shell command line - simply checking e.g. a "curl https://www.google.com" will tell you. The cURL is usually part of the device OS but for some device types it may need to be installed separately. Agents that are in the EXPERIMENTAL state may require a user to perform the installation manually (consult the README file inside the agent archive).
  4. If all the above is correct, you may look inside the agent log files. Any error should be reported there.
    1. UI: in case the agent type you are using does provide a web UI, look for System Logs (usually under Status)
    2. Device's filesystem: the log is situated here: /opt/adwarfg/global_log.txt for general log and for every agent run here: /opt/adwarfg/<PID>/logfile.txt (you need to replace "PID" by the actual process id)

Licensing issues

Licensing issues can be quickly identified by looking at the Dashboard page.

Number of devices

The light just next to the global light is showing the number of free slots in your license. If the light is yellow, it means your at exactly on the license capacity and any new connecting device will be declined. If the light is red, it means there were some new registration requests on top of the already full license and the (negative) number is showing the count of declined registration attempts since Dwarfguard service startup. If you have some devices that are no longer connected to Dwarfguard, you may open their Details page and delete them to recover free slots. If you want to increase the number of slots, you need to request new license (contact your dealer or DT) or update the SaaS conditions (contact your service provider). You may want to visit Dwarf Technologies Contacts Page or Distribution Partners Page

License validity period

The second light next to the global light shows the number of days until your license validity period expires. It turns yellow when 20 or less days remains and it turns red once the license expires. If you are on SaaS, it should never turn red as the service provider is in charge of your license validity and it's his responsibility to provide a valid license for you - contact your service provider in case of a license expiration problem on SaaS. If you are using a purchased license, see either your dealer or DT contact. You may want to visit Dwarf Technologies Contacts Page or Distribution Partners Page

Non-critical problems

Most of the non-critical problems can be quickly identified from the Dashboard. If you are unsure what the problem is, read on. If you already know what problem are you experiencing see possible problems resolution below the Identification from the Dashboard section.

Identification from the Dashboard

When you open the Dashboard page and the first (biggest) light with caption "Global" is shining yellow, follow the points below. If it is shining green, it means that Dwarfguard does not detect any problem on your devices so you will not be able to identify problem from the Dashboard. If you are unable to add devices to Dwarfguard, please see the section Problems adding devices to Dwarfguard above. Alternatively, you may browse through the possible problems resolution below this section.

  • The caption directly below the main light should give a clue where is the source of the problem.
  • If not clear, orient yourself based on which of the additional four lights is not green:
    • The first light represents number of free device slots (allowed by license). In case there are no free slots but no new device registration has been denied, the status is yellow (warning). In case there were registration requests from devices after all the free slots has been taken, the status is red and the counter shows number of declined registration requests as a negative number (since Dwarfguard service start). See Licensing Issues, Number of devices above.
    • The second light represents license status. If your license period is expired, the light goes red. If your license period is about to expire, light goes yellow. See Licensing Issues, License validity period
    • The third light captures status of tunnels. If a device does not respond to the tunnel request, light goes to yellow or red. Please inspect table of Webtunnels (Tunnels -> Web) and reset or remove the malfunctioning tunnel. Once removed, you may attempt to setup the tunnel again.
    • The last light covers the status of devices - the worst state of all the monitoring groups. If there is for example a triggered alert for some device, the light goes from green to either yellow or red, depending on the particular situation. The fastest way to locate the problem is to scroll down the dashboard and look for the monitoring group causing the problem. Following covers the usual culprits:
    • If the problem is caused by an alert being triggered, you just scroll down below the groups. Each triggered alert is listed there with the breakdown of the devices and a link to the list of devices. Note that alerts that are defined on a group are reported against that group. Alerts that are defined on a specific device are reported against the All devices group.
    • If the problem is caused by the device being in non-synced state, your fastest way to find such device is to going to the Devices menu and filtering the devices based on the Sync status. This way only the devices with the selected synchronization status are shown.
    • If the problem is caused by devices not reporting to Dwarfguard, your best way to locate such devices (in version 0.6 and below) is to look at the Late device alert that is a shipped (default) alert. All devices that are late (more than 20 missed contacts) are reported by this alert. If the shipped alert is disabled, you just need to enable it and in under 5 minutes it will report all such late devices. If you want to report the Delayed devices, the best way is creating a custom alert for e.g. more than 1 missed contacts and setting that alert on monitoring group "All devices".

Resolving the cause of the problem

Please note that problems with adding a device to Dwarfguard and licensing-related problems are discussed above.

Accessing Dwarfguard

To be able to access Dwarfguard, you need: - correct URL - access to Dwarfguard URL (Internet for Internet accessible deployments, private cloud/VPN for private cloud/private network deployments) - correct credentials - Dwarfguard must be both running and ports not blocked by another service

If you cannot see the login screen, make sure you have the correct URL and that the server is up and service running. You may simply connect to one of the connected devices and look up the URL/IP address. If the device is connecting to the server successfully, you just need to take the URL/IP and use it. In case of UI-enabled agent (e.g. Advantech cellular router agent), you can also see if the healthcheck is passed which gives additional information.

Regardless of agent type, you should be able to find the URL/IP address either from /opt/adwarfg/server.ini file or, if the file is not present look for the dwarfg_agent.sh script (either in /opt/adwarfg/ or /opt/adwarfg/bin/) and see what are the values for variables SERVER and DEF_SERVER (take the first defined).

Once you are sure you have the correct URL, put it into the browser and try to connect. If you still do not see the login screen (but the connected devices are working fine), you need to check your network routing - chances are that the devices are accessing the server via a different path that is enabled but at the same time, the network route you are using is not allowed. This is a network configuration issue outside of Dwarfguard and needs to be resolved on a network level. If you are using SaaS, please contact your Dwarfguard SaaS service provider. If you are using managed service, please contact your Dwarfguard administrator.

If you have no networking-related problem but the login screen is still not showing up, it means that the server configuration is wrong. Either the deployment was not finished correctly or something is blocking Dwarfguard web from working. Please note that Dwarfguard service and web UI are two separate components and the web component should be accessible regardless the service being running or stopped. If you are using SaaS, please contact your SaaS service provider. If you are using managed service, please contact your server administrator.

In the case you can see the login screen but are unable to log in, there are a possible issues:

  1. Incorrect credentials. NOTE various causes:
    • The credentials has been changed.
    • You are connecting to a different server / deployment.
    • The server has been reinstalled.
  2. Incorrect server
    • The URL you have provided leads to a different server / deployment than you think (wrong URL).
    • Server is housing several Dwarfguard deployments and due to misconfiguration you are accessing different deployment (correct URL, configuration error during deployment).
  3. Resources problem
    • Server is having resources outage (e.g. the server is running multiple VM containers and having out-of-memory issues)
    • Server is experiencing HW issue

In any of the cases above, you should escalate the problem to the server administrator. If you are using SaaS, reach to your service provider.

Reporting the problem

Instructions on what to do to properly report the problem and thus make problem resolution as fast as possible:

  • Have you looked at Critical problems section (if having a critical problem)? If so, please describe (when raising the issue with support) what you can and cannot access.
  • If you do not have a critical problem, have you looked at Identification from the Dashboard? If so, please describe (when raising the issue with support) what problem are you experiencing and what are your expectation. Also, as you are able to access UI, please include logs in your issue report:
    • Visit System -> Event Log
    • Click Download Log button (next to Event Log page caption on the top)
    • The most relevant information will be provided in the form of .tar.gz archive. Store it and have it ready to attach to ticket you are going to open.
    • While the logs do NOT contain sensitive information like passwords or license, some information like IP addresses may be part of the log, so you are still discouraged to share the logs in public. (feel free to examine the log file to make sure what is and what is not included). The intended usage of the logs is providing the information archive for the purpose of problem analysis and resolution to either your support provider or Dwarf Technologies.
  • Follow your support link that can be found either in Help -> Troubleshooting close to the text beginning or directly in the license (System -> License Management, pane Active Licence Information, field Support link:)
  • Proceed according to your support provider instructions.
    • In case your service provider is us (Dwarf Technologies), please follow directly ticketing system, authenticate using your credentials obtained during license purchase and open incident report.

Typical Dwarfguard administration tasks (for self-managed deployments)

Please follow to Dwarfguard Administrator's notes

Copyright (c) 2023 by     Wiki is powered by Fossil SCM