On 2003 June 23, all of NCF's analog modems and analog voice lines were taken out of service, and digital modems with digital PRI lines were installed. Reliability (and service) improved dramatically. It's no longer a daily chore to reset a modem to get it back into operation. Thus this log has been discontinued. However, to remind us of the 'good old days', we've kept the log below around.
The material below is NOT CURRENT and NOT MAINTAINED.
This web log records changes made to the NAS units.
NCF has (used to have) several NAS units at Carleton University, numbered 1 to 'N'. Each NAS unit has a shelf of async cards, and each card has four ports (modems). Depending on the NAS, a shelf may hold up to 48 ports, numbered starting at 1. Each port is connected to a phone line, identified by an arbitrary label so that it can be tracked (in case there are problems associated with a particular line). Typical line labels are "9013-45" or "Q" or "23".
A quad modem card has two parts -- the Network Interface Card (NIC) which has connectors and modems for four phone lines, and the Network Application Card (NAC) which handles session protocol. Both the NIC and the NAC need to work to get a working port on an analog phone line (with digital lines, the NIC part isn't required). The trick is to match working NIC's with working NAC's (or more to the point, to line up bad ports on a NIC with bad ports on a NAC).
Port activity is monitored by NCF's Modem Sharing system. Each night a line is added to the Port Report that describes activity observed during the previous day. Ports that have unexpected inactivity or unusual session lengths are suspected of being faulty or suspected of being connected to a faulty phone line.
Configuration errors occur when the actual port and line assignments are out of sync with what the Modem Sharing system has been told (by information in configuration files).
|2||1, 4, 10, 14, 15||3, 6, 7, 8|
|3||6, 13||5, 11, 15|
Port 3.16 (office/demo port) is normally connected to a test line; inactivity on that port does not indicate a fault.
Line 13 on 3.16 (office port) is waiting for a port. Move it back to 5.33 when line A (now on 5.33) can be placed on a port.
Office line is dangling ('ring no answer'), waiting to get back on 3.16.
On busy: Line 5 (1135)
Moved line 9013-57 from 4#34 to busy-out
Moved line from NAS6#48 to busy-out
AD: NAS 4#25 answers but fails to negotiate. Reset card to no avail. Moved line 9013-57 from 4#25 to 4#34.
AD: NAS 5#19 RNA. Reseated card with ports 33-36, and ports are now answering.
AD: NAS 1#1 RNA. Had left the demo line plugged in since yesterday.
AD: NAS 1 Port 1 busy. Reseated card.
AD: NAS 4 Port 33 RNA. Reseated card with ports 33-36, and ports are now answering.
AD: NAS 3#28 busy, NAS 5#28 RNA. Reseated cards, ports now answering.
AD: NAS 3 Port 9 giving busy-out. Reseated card with ports 9-12, and ports are now answering.
AD: Card 13 of NAS 4 is recognized by the NAS but none of the ports will function. Reseating the NAC and NIC had no effect. However, the pair of cards function when moved to slot 1 (normally used by PRI). Tried the pair of cards in slot 15, but did not work. Moved the pair to slot 15 of NAS 5, where they worked.
Slot 15 of NAS 5 appears to NAS 5 as ports 77-80. ModemServer config was updated to accomodate this (and four ports were removed from NAS 4). The following lines are attached to NAS 5, card 15 (ports 77-80): 9013-33, 9013-34, 9013-35, 9013-36.
Card 1 of NAS 1 was reseated (to clear a fault on port 1).
Line 9013-22 was moved from busy to NAS 3.13.
JS: To clear some port faults, reseated cards 12 and 13 in NAS 4, and card 3 of NAS 2.
ModemServer config updated to match new Carleton pool arrangement (all ports in one pool, with the pre-existing four phone numbers all entering the same pool).
Mitel port 1.29 (shown as inactive). Port status is 'autoselect'. Reset command clears it, but the fault just returns. Will use dialupTester to determine what the effect of the fault is, and thus the urgency of going out to the site to correct it.
Mitel port 1.29 (shown as inactive). Port status is 'autoselect'. Reset command cleared it.
Port 3.10 (shown as inactive). Port status is 'autoselect'. Reset command ineffective. Actions: reseated card -- fault cleared.
Port 5.11 (shown as inactive) stopped responding Sunday. Ring-no-answer, does not respond to software resets. It is a port on one of the new cards. Action: Card pulled and re-inserted. Tested, working. Cause of fault unknown.
NAS 2 not operating -- evidently it was disrupted by hot-swapping cards. Action: Reload and resets had no effect. Power cycle at 11am restored it. Other changes: Hard session limit of 60 minutes was discovered on NAS 1 to 3; removed (should be under modemServer control, not NAS). Logging to syslog f4:local5 enabled.
Reboot of tc1 and tc2 did not affect the dead ports on those NAS.
Installing two new quad cards (two NAC and two NIC). New NACs labelled A and B, new NICs labelled 2 and 3. Installed them in spare slots of USR4. Two of the eight ports do not work. By swapping NICs, it appears the NACs are OK but each NIC has one bad port. Taking this opportunity to test a few NICs, here are the results:
Good NICs from USR2/USR3 were moved to USR4 for the new NACs (A and B). NICs 1, 2, and 3 were matched with NACs with corresponding bad ports on USR2 and USR3. The net effect was to gain one port plus all eight new ports on NACs A and B.
Line 9013-19 on 2.3 moved to 3.14 (a newly-recovered port).
Line 5 moved from 5.5 to busy.
Line 9013-17 moved from busy to 4.5.
Line 9013-6 moved from busy to 4.6.
Line 9013-49 moved from busy to 4.7.
Line 9013-16 moved from busy to 4.8.
Line 9013-74 moved from busy to 4.9.
Line 9013-1 moved from busy to 4.10.
Line 9013-31 moved from busy to 4.11.
Line 12 (1135) moved from busy to 4.12.
Configuration changes: Port 4.5 to 4.12 enabled in pool 2; port 3.14 enabled in pool 1.
Port 5.5 (shown as inactive) on line 5 stopped responding Friday 11am. Software and hardware resets have no effect. Test line was attached and port is returning 'busy'. Action taken: Since there are no more busy-out connectors, leave line 5 attached to port 5, configure 5.5 as idle.
Port 6.8 (shown as inactive) on line 12 stopped responding Sat 2pm. Software and hardware resets have no effect. Test line says port is 'ring no answer'. Action: Move line 12 to busy. Move line 13 from port 5.33 to 3.16 (TEMPORARY). Move line A from busy to port 5.33. (Line A wasn't long enough to reach 3.16, so a line that could reach 3.16 (line 13) was moved there, and line A filled in for line 13.)
Port 2.6 (found inactive) on line 9013-19 stopped responding at Mon 11am. Action: Move line 9013-19 to port 2.3, which returns busy. 2.6 configured as idle.
A more visible label was attached to the test/demo line.
Port 2.4 (shown as inactive) on line 9013-17 stopped responding Saturday afternoon. Software resets have no effect on the port. Action taken: Line 9013-17 to that port was busied out. A trial with the test line indicates that the port is returning 'busy'. Changed the config of 2.4 to idle.
Lines on busy: 9013-1, 9013-6, 9013-16, 9013-17, 9013-31, 9013-49, 9013-74, "A"
10am: DialupTester reports problems on 520-9013 but no problems on 520-1135. This is consistent with this morning's activity report. Here are the changes made around 10am in reaction to this morning's activity report:
There are now 6 lines on busy.
modemServer restarted around 10am to make the changes effective.
Moved lines off ports that are marked as problems in the ModemServer Port Report for Nov20. There are not enough known-good ports to service all the lines, so the extra lines are busied out (made permanently busy) until new equipment can be found. If the 1130/1135 lines on Total Control NAS's can pass dialupTester with no faults, we'll replace NAS 1-3 with a new Total Control NAS.
Ports that were unexpectedly inactive could be bad or could have bad lines attached. To distinguish those cases, the lines on those ports will be moved to known good ports. If the inactivity appears on the new port, we'll know it was the line, not the port.
Four lines are now temporarily 'permanent busy' until we can get more known-good ports to service them.
Updated the configuration file and restarted modemServer at about 2pm to make the file effective.
ModemServer configuration file updated with recent NAS port assignments. Let's see if ModemServer's port report will identify weak ports or lines.