© 2009 Avaya Inc. All rights reserved.1 Chapter Nine, Troubleshooting Module One – Using Diagnostic Tools

© 2009 Avaya Inc. All rights reserved.2 Avaya Diagnostic Tools Depending on the nature of an issue, there are several tools that may be useful for troubleshooting problems. For example, Ping, Trace Route and Netstat are useful for troubleshooting network connectivity issues. Additionally, Avaya provides three diagnostic tools for use specifically with IP Office: Debug View – used with Voic Pro System Status Application - a diagnostic tool for system managers and administrators, in order to monitor and check the status of IP Office systems System Monitor – used by system managers and administrators in order to capture system traces for submission to developers and support staff

© 2009 Avaya Inc. All rights reserved.3 Avaya Diagnostic Tools installation Installation – The System Monitor and System Status Application are provided with the IP Office Admin Suite.

© 2009 Avaya Inc. All rights reserved.4 IP Office Debug View The voic console mode (Interact with Desktop) is no longer used and Avaya recommends using the Debug Viewer to view voic server activity. Can be used to view server activity real time or logged to file Reporting detail level can be configured Used for tracing various functions including: – Voic Connections – Outcalling – Message Processing Debug View must be run on the Voic Pro server PC

© 2009 Avaya Inc. All rights reserved.5 Debug View Logging Options By default events are shown in the DbgView window. The output can also be logged to a specified file. – Select File | Log Preferences from the DbgView menu – Specify which Log Mode to use – Enter the name of the file to store the information The log file is best viewed with WordPad word wrap disabled

© 2009 Avaya Inc. All rights reserved.6 Debug View Filtering The level of filtering can be set from 0 – None to 9 – High Trace to control the level of detail. Specific items can be selected or deselected to target specific output message types (e.g. TTS Support or MAPI Support)

© 2009 Avaya Inc. All rights reserved.7 Starting System Status Application (SSA) SSA can be launched independently or via Manager. For IP500v2 systems, it can also be launched by browsing to the IP Office IP address without needing to install the application. – Click the Windows Start icon and select Programs | IP Office | System Status – From within the IP Office Manager application, select File | Advanced | System Status – Using a browser, enter the IP address of the IP500v2 system, for example – The Offline tab of the SSA Login provides the option to open previously saved files such as a Snapshot or Continuous log.

© 2009 Avaya Inc. All rights reserved.8 Services Base TCP Port The port(s) on which the IP Office listens for requests to access services on the LAN1 IP address. Each service uses a port offset from the base port value. If this value is changed from its default, the Manager application must be set to the same Base TCP Port through its Services Base TCP Port setting (File | Preferences). The port is set on the System Details screen within the IP Office Security Settings (File | Advanced | Security Settings).

© 2009 Avaya Inc. All rights reserved.9 Assigning Security Rights The default IP Office service users Administrator, Manager and Operator have SSA access rights. There can be up to two SSA clients connected to an IP Office unit at one time. The CCR application uses the same interface to the IP Office control unit as SSA.

© 2009 Avaya Inc. All rights reserved.10 System Status Application (SSA) SSA provides information on the following: – Alarms SSA displays all alarms which are recorded within IP Office for each device in error. The number of occurrences and the date and time of the last occurrence are recorded. – Call Details Information on incoming and outgoing calls; including call length, call reference and routing information. – Extensions SSA details all extensions (including device type and port location) on the IP Office system. Information on the current status of a device is also displayed. – Trunks IP Office trunks and connections (VoIP, analog and digital) and their current status are displayed.

© 2009 Avaya Inc. All rights reserved.11 System Status Application - continued Resources IP Office includes central resources that are utilized to perform various functions. Diagnosing these resources is often critical to the successful operation of the system. Those resources include: – Voic Channels – Conference Channels – Data Channels – VCM Channels – Modem Channels

© 2009 Avaya Inc. All rights reserved.12 System Status Application - continued Voic SSA provides real time information on the number of licenses available and details of the voic mailboxes including the number of messages and the current status of basic voic settings for the associated mailbox user or hunt group. IP Networking SSA displays the details of the IP Office Routing Table, including static and RIP routes as well as information on configured IPSec or L2TP VPN tunnels. – A Ping can be executed from the IP Routes screen choosing LAN1, LAN2 (WAN) or RemoteManager (DialUP) as the source interface to aid in troubleshooting connectivity issues.

© 2009 Avaya Inc. All rights reserved.13 System Status Application - continued System The System screen within SSA provides information on the type and state of hardware installed in the IP Office to include: – Hardware Summary – Memory Cards – Control Unit Ports – Extension Ports – Expansion Modules – H.323 Extensions – VoIP Trunks

© 2009 Avaya Inc. All rights reserved.14 SSA Snapshot Feature Snapshot only captures the IP Office system status to a file with the.ssh extension which can be viewed offline at a later time. Include switch configuration: embeds a copy of the IP Office configuration file when Snapshot only or Continuous log is selected. Continuous log : records all IP Office activity to a file or series of files with the.slo extension until logging is stopped. If logging is enabled for an extended period, new files are created based on the settings Daily at midnight or After the log reaches a configurable size.

© 2009 Avaya Inc. All rights reserved.15 Reviewing Files Offline Using the Offline tab of the SSA Login screen provides the option to review previously saved files with the.ssh or.slo file extension. When using SSA to open a Continuous log file, the Replay options are available to control the log file playback. The Manager application can be used to review the IP Office configuration embedded into the SSA log files with either the.ssh or.slo file exensions.

© 2009 Avaya Inc. All rights reserved.16 SSA Tracing Options The System Status Application provides the ability to generate traces for calls, extensions and lines / trunks: Calls: From the Call Details screen the trace will show the changes of state for that call and events relating to both ends of the call. Extensions: From the screen for a specific extension, the trace will show events relating to the extension including button presses. Trace information for a call from a specific extension will provide the same detail as the Call Details screen for the same call. Trunks: The Trunk screen will show events relating to the selected channels, such as protocol messages, plus traces of all calls associated with these channels, for as long as the calls are connected.

© 2009 Avaya Inc. All rights reserved.17 SSA Tracing Options - continued Trace information is presented at the bottom of the screen. The Pause button and scroll bar enable viewing the information while the application continues to record new trace events. The Resume button displays all the events recorded when a trace is paused, as well as further new events as they occur. When a trace is displayed, the option to Print and/or Save As are available. A trace can be saved to file either as a.txt or.csv file. – If the trace is paused, only the information currently displayed will be saved and/or printed. To diagnose problems with a call, it is generally best to trace the source of the call; e.g. trace the trunk for an incoming call or the extension for an outgoing call.

© 2009 Avaya Inc. All rights reserved.18 Using System Monitor The Password used for System Monitor is set in the IP Office system Security Settings. If the IP Office doesnt have a Monitor Password set, the System Password is used. By default, the System Password is password (case sensitive).

© 2009 Avaya Inc. All rights reserved.19 Starting System Monitor System Monitor can be run on a PC on the same local subnet as the IP Office or from a PC on a remote network with established bi-directional routing. Click on Start | Programs | IP Office | System Monitor Enter the IP address of the IP Office control unit Enter the Password and click OK

© 2009 Avaya Inc. All rights reserved.20 System Monitor Interface The IP Office System Monitor application is used to assist in the diagnosis of problems. Through configuration of its settings it is able to display information on a specific area of an IP Office's operation. It can capture that information to log files for later analysis.

© 2009 Avaya Inc. All rights reserved.21 System Monitor Filters / Trace Options The System Monitor can be set to display and log a wide range of traffic types on the IP Office. In order to avoid missing significant events due to The Filters | Trace Options menu allows for customization of the events to trace. All IP Office installers and maintainers need to understand how to run Monitor when necessary as Avaya may request copies of Monitor traces to resolve support issues.

© 2009 Avaya Inc. All rights reserved.22 All Settings Options On the bottom of the All Settings screen there are buttons to aid in the selection of options. Additionally, if a combination of settings is configured to troubleshoot a specific issue, the settings can be exported using the Save File button for later use. On occasion Avaya support staff may provide specific settings which can be imported using the Load File button.

© 2009 Avaya Inc. All rights reserved.23 System Monitor File Logging Logging is enabled for the System Monitor application from the menu File | Log Preferences. Select the options for logging and click OK.

© 2009 Avaya Inc. All rights reserved.24 Log File Naming Convention The naming convention is as follows: NAME DATE TIME (N) – In the above example, the NAME of the trace is CallTrace1 – The DATE format is: YYYYMMDD where Y represents the Year, M represents the Month and D represents the Day of the Month – The TIME format is: HHMMSS where H represents the Hour, M represents the Minute and S represents the Seconds – The (N) represents the Number of times the log file of the same name has rolled over either manually or based on the Log Mode settings enabled.

© 2009 Avaya Inc. All rights reserved.25 Event Notification The IP Office supports a number of methods by which events occurring on the system can be reported. These are in addition to the real-time and historical reports available through the IP Office System Status Application (SSA). SNMP Reporting: The IP Office SNMP client allows the IP Office system to respond to SNMP polling and to send alarm information to SNMP servers. SMTP Reporting: IP Office can send alarms using an SMTP server. Syslog Reporting: IP Office can also send alarms and audit trail events to a Syslog server (RFC 3164) without needing to configure an SNMP server.

© 2009 Avaya Inc. All rights reserved.26 SNMP Reporting Support for SNMP is enabled by entering the required information on the System Events | Configuration tab. – Community (Read-only) and SNMP Port – Enabling SNMP is not required if using SMTP or Syslog. The QOS Parameters are used if Enable RTCP Monitor on Port 5005 is selected (System | LAN1 | VoIP). If the thresholds are exceeded an alarm is sent to SSA and to a destination configured on the Alarms tab.

© 2009 Avaya Inc. All rights reserved.27 SNMP Settings Enter the IP address of the SNMP server (Trap Receiver) Verify the Port setting (162 is the default transmit port) Enter the Community [string] configured in the SNMP server

© 2009 Avaya Inc. All rights reserved.28 SMTP Settings In order to send alarms to , the IP Office Manager System | SMTP tab must be configured with the senders information. Depending of the authentication requirements of the SMTP server, this may need to be a valid address hosted by that server. Otherwise the SMTP server may need to be configured to support SMTP relay.

© 2009 Avaya Inc. All rights reserved.29 Syslog Reporting Enter the IP address of the Syslog server Verify the port setting (514 is the default port used by Syslog)