Objective
Understand how to read and interpret Network Utility results coming from the Network Utility Tool.
Applies To
- Networking
- Network Utility Tool
Procedure
The Network Utility provides a snapshot of network settings and diagnostic results. While the test is comprehensive some network settings that affect phone service may not be captured in the test. In the event of issues occurring that are likely related to network configuration, the firewall configuration may still need to be reviewed despite passing the Network Utility test
Network Utility Results have the following categories:
- DNS Testing
- Ping Test
- NAT
- Outbound Connectivity (See Required Ports listed in the X Series Technical Requirements)
- System Info
- ALG Test
- Media Testing
- Fragmentation Testing
- Trace Route
- Bufferbloat
DNS Testing (NetUtil version 2.3)
DNS Testing tests the network's DNS against the 8x8 GTM (8x8 DNS) to verify devices are geo-routing to the correct Region. This testing is particularly useful for troubleshooting Line Unregistered for SRTP-enabled lines, and call quality issues.
8x8's Global Reach technology will return multiple data centers in a single region to ensure stability in the event of an outage by making the devices aware of several different resources in that region. This is why when looking at line 35 and 39 you will see U.S. East and U.S. West show up. This is perfectly fine and expected. It is also expected that the IPs and the order of the IPs returned in a FQDN query/result will change.
To identify geo-routing issues look through the DNS test results and locate the following message:
*** Conflicting DNS resolutions: Two or more DNS Servers are in disagreement over 'pcsbc.packet8.net'!
As an example this user has a routing issue to the Poly SBC as well as the Secure SBC (Extensions with SRTP enabled):
The location's local DNS identified as: "[Local DNS Server 1]" is routing to the India Region, and the GTM (8x8's DNS) is routing to the USA Region. This user happened to be located on the US. Using this information, we can see the user is geo-routing incorrectly using their DNS to the India Region. To resolve this the user will need to change their DNS in use on the network or have the Poly devices use the GTMs as their DNS Server.
Additional information on DNS and Geo-Routing
- DNS Result Destinations (See What are the Domains on the Network Utility DNS Test used for?)
- Georouting works on the public IP of the DNS server. MPLS, VPN's, and hosted DNS servers can falsely report geo-routing issues. It may be beneficial to review 8x8 Work Analytics call quality reports to verify where the calls are truly routing.
- DNS Servers that make use of Anycast IPs are not recommended as this will result in using DNS Servers outside your geographic area and result in incorrect Geo Routing.
- The continuous test should be used as a source of information that an issue is present. Once an issue has been identified agents should move on to tools such as Ping Plotter to determine the source (ISP or LAN).
- The Local DNS Servers in the NUT test are taken from the Preferred DNS server and Alternate DNS server settings specified in the OS running the tool. If none are specified, it will use the server provided by the router. This may lead to results that are not relevant, for example, when troubleshooting desk phone issues.
Ping Test
The ping test sends ping requests to each data center to check for high latency and/or packet loss over the connection. This test will not determine where the issue is coming from, only that a potential issue is present.
Use WinMTR to get more detailed information on latency, packet loss and a potential cause for high latency and/or packet loss issues if the problem is consistent. If the issue is not consistent use Ping Plotter for an extended test with time stamps. These tests will help determine where an issue is coming from if one is present.
The test below shows that ping requests within the United States, Canada, United Kingdom were fine but a warning message was provided for Hong Kong, Australia, Singapore, Amsterdam, and China. Only results from your geographic region should be considered, for example, you are in Ohio thus Only US East and US West coast data centers should be considered.
NAT
The NAT test only determines the NAT type. This does not indicate any issues with the network as each firewall and router handles NAT differently. Not all NATs are properly identified, if there is an error, this may be ignored.
Additional information on NAT and NAT Types
Full Cone NAT (Static NAT)
Full cone NAT (also known as one to one NAT) is the only type of NAT where the port is permanently open and allow inbound connections from any external host. A full cone NAT maps a public IP address and port to a LAN IP and port. Any external host can send data to the LAN IP through the mapped NAT IP and port. This type of NAT is also known as port forwarding. This is the least restrictive type of NAT as the only requirement is the connection comes in on a specific port - the one opened by the endpoint.
Restricted Cone NAT (Dynamic NAT)
A restricted cone NAT works in the fashion as full cone NAT, but adds additional restriction based on an IP address. The internal client must have first sent packets to an external host before it can receive packets from the external host. In terms of restrictions, the only requirement is that the packets come in on the mapped port and from an IP the internal client has sent packets to.
Port Restricted Cone NAT (Dynamic NAT)
Port restricted cone NAT functions the same as Restricted Cone NAT, but applies restrictions to ports as well. Restricted cone NAT will accept connections from any source port, a port restricted cone NAT restricts this further by only accepting connection from the IP and port it sent an outbound request to.
Symmetric NAT (Dynamic NAT)
Symmetric NAT applies restrictions in the same way as a port restricted cone NAT, but handles NAT translation differently. Symmetric NAT has its own unique issues. These issues stem from generating randomly generated ports and applying them to the client when reaching out to destinations. This NAT type does not use port preservation.
Outbound Connectivity
Outbound Connectivity testing indicates if ports are being blocked on the network, either individual ports or ranges on the firewall or router. If ports are blocked this could cause connectivity issues or dropped calls depending on what ports are blocked. The firewall should be configured per our guide on X Series Technical Requirements allowing ALL ports listed in the Required Section, optionally add UDP 3478-3480 which is used by the network utility test's media, fragmentation and bufferbloat tests.
Below is an example of a test where ports are blocked:
In contrast, below is an example of a test where all ports are available:
Additional Information on Outbound Connectivity Test
- Endpoint security applications (Anti-Virus or PC firewall applications) can cause this test to fail. It is suggested to disable these applications or add an exception in the application when running the network utility.
- This test may fail due to the router or firewall on the LAN having issues. In cases where there are no ports to open on the device (such as working with a Netgear), restart the router and retest. If the test continues to fail there may be an issue with the device itself (such as it's failing) or there is a firmware issue.
- This test can potentially fail if the subnet the connectivity test is running to is not white listed on the firewall. The location the customer is geo-routing to will be displayed at the top of the test as follows: Running Outbound Connectivity Test on 'netutil.8x8.com' The IP address will change due to destination/region.
- Failures may be returned from any Layer 2 or 3 devices in your network including access points, or if you have multiple firewalls in your network.
System Info
Provides information about the computer that is running the test. This is especially useful for users running only 8x8 Work Desktop. We can determine if the tested machine is meeting the minimum system requirements or is in an unsupported environment. The results will change based on the Operating system (OS), and OS Version as well as the device.
Windows
MAC
Additional Information on System Info
It is recommended to identify any potential VPN connections on the system information test. Additionally, we want to make sure the test is run on the same network as the phones.
ALG Test
ALG testing determines if ALG is enabled on the LAN. If ALG is enabled it can cause a multitude of issues such as dropped calls, echo, call termination, one-way audio, and no audio. If the test fails you will see the following message:
In contrast, here is an example of an ALG test with no errors:
Additional Information on the ALG test
- The ALG test verifies the checksum passed in the message at the server and at the client.
- If the checksum does not match the application will return "ALG detected" either at the server or at the client.
- If the message is not returned or a time out is received this typically indicates blocked ports or the message was modified and was dropped.
- When the time out message is received or no response is received the user will need to disable any endpoint security and verify ports 5060 and 5199 are allowed.
- ALG may be enabled on your carrier's modem/router and may require you to contact your carrier to disable it.
Media Testing
The media testing makes a series of mock calls to each data center. From this information, we can determine if there will be potential issues on calls due to packet loss, latency, or jitter. If the results are showing a warning or failure, the user may have call quality issues. Keep in mind this data is only relevant to the closest data center to the tested location, as well as the backup.
The test below shows that media tests within the United States, Canada, United Kingdom were fine but a warning message was provided for Hong Kong, Australia, Singapore, Amsterdam, and China. Only results from your geographic region should be considered, for example, you are in Ohio thus Only US East and US West coast data centers should be considered.
Fragmentation Testing
Fragmentation Definition:
Fragmentation is a technique that divides a data packet into smaller data packets so that they can be sent through a network that can only transfer small data packets. Fragmentation occurs during network transmission. When these packets are received at their destination, they are reassembled to their original data packet size.
Supporting Fragmentation:
In rare situations, signaling packets may become extremely large and require to be fragmented. 8x8 recommends that Fragmentation be supported by all network equipment, including routers. Fragmentation is NOT responsible for any audio quality issues.
If Fragmentation Fails:
We did find that Fragmentation may not be supported by your edge device. In rare situations, signaling packets may become extremely large and require to be fragmented. If fragmentation is not supported there will be a loss of audio. 8x8 recommends that Fragmentation be supported by all network equipment, including routers. 8x8 also recommends that your MTU be set to 1500 on all networking gear.
The test below shows that fragmentation tests within the United States, Canada, United Kingdom were fine but a warning message was provided for Hong Kong, Australia, Singapore, Amsterdam, and China. Only results from your geographic region should be considered, for example, you are in Ohio thus Only US East and US West coast data centers should be considered.
Traceroute
This test runs a traceroute to the nearest data center to the tested location based on geo-routing information. If there is a consistent issue along the path to the data center you will be able to determine where it’s coming from on the traceroute.
This test is also useful for identifying the route the connection is taking from the local network to the path out to the public internet.
Additional Trace Route Information
- Keep in mind, if a machine is using an MPLS connection, a VPN, or multiple VLAN's the results seen on the test may not reflect as an accurate representation of a local issue. It is suggested to verify with the locations I.T. consultant how the machines are connected on the network.
- Some ISP's may have nodes that are not configured to show the public IP address on the route out over the public internet. It is possible that there may be local IP's reflected on the traceroute that do not indicate an issue.
- ICMP may be blocked by your firewall or not supported by a particular hop/carrier, in this case, the test can be ignored, it is purely for helping identify the route taken if there are issues.
Bufferbloat
Bufferbloat is the undesirable latency that comes from a router or other network equipment buffering too much data. Below is an example of a network with bufferbloat:
An example of a network without bufferbloat:
Additional Information on Bufferbloat
- This test does not always complete properly and may not be a good indication of issues on the LAN
- If excessive bufferbloat is detected it is suggested to power cycle the network and re-run the test to determine if the results were correct.
- If the test fails, there may be a potential firmware issue causing memory leaks and excessive buffering on the device. The user should update or roll back the firmware version they are using on their device and re-test.
- As a last resort, contact the device manufacturer for a solution to this issue. Although in all likelihood the device will need to be replaced at this point.
Additional Details
For troubleshooting assistance, see the troubleshooting section on the download page for Network Utility Download.