David's Blog

The CallManager system I look after recently developed an odd issue. Faxes had been working quite happily for some time, but suddenly they were failing left, right and centre. Not only that, we had quite a few credit card machines running over VoIP which were failing to take payments. All ran via Cisco ATA 186 analogue gateways.

I ran various tests and found that faxes between ATAs internally were perfectly fine. As soon as they went through the gateways, they failed miserably. I ran through some of Cisco’s help guides but was drawing complete blanks. I’d ran the prserv tool to see what the ATAs were up to whilst making calls. I’d see ‘resync’ go whizzing by with a load of other seemingly random numbers. The word ‘resync’ suggested to me that the ATA was hiccuping on something, and doing something to the audio stream.

Analogue modems expect a constant stream of data. It might get fuzzy, or drop out, but it will always come along in a specific order, at a certain time. It’s predictable. Resyncing something mid-stream isn’t a good idea to a modem. In all cases when I saw the word ‘resync’ the fax would end up corrupted or dropped entirely, depending on when it happened during the call.

I did a little digging on the Cisco TAC case collection, and found what I was looking for. It was something I fiddled with some time ago.

ISDN circuits rely on a clock. Generally speaking, the clock is the telco end. We have four ISDN links on our gateways – two out to the PSTN, and two QSIG links to the old PBX. We had some odd issues with echo, and one thing we tried was forcing clock sync on the E1 controllers.

E1 and T1 controllers can exhibit something known as ‘slipped seconds’. This is basically where the clocks at both ends get slightly out of sync with eachother. In some instances it can cause echo, so we’d nailed up the QSIG links to use the clock at the legacy PBX end. However, it seems with the 2-port WICs, this causes BOTH ports to sync to that clock.

Up to this point it hadn’t been the issue. There was the odd dropped fax, but nothing overly bad. A week or so ago (the week I was off, natch), the faxes all pretty much failed simultaneously. Voice calls remained perfectly fine, which made it all the more perplexing. Luckily I found the info I needed in the TAC collection.

The issue will manifest itself as slipped seconds. On the router, I did the following:

router#sh controller e1 0/0/0
E1 0/0/0 is up.
  Applique type is Channelized E1 - balanced
  No alarms detected.
  alarm-trigger is not set
  Version info Firmware: 20060707, FPGA: 13, spm_count = 0
  Framing is CRC4, Line Code is HDB3, Clock Source is Line.
  CRC Threshold is 320. Reported from firmware  is 320.
  Data in current interval (618 seconds elapsed):
     0 Line Code Violations, 0 Path Code Violations
     181 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     181 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
  Data in Interval 1:
     0 Line Code Violations, 0 Path Code Violations
     262 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     262 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs

There we see some ‘slip secs’.

The ‘fix’ is quite simple. I switched to the interface, and then issued the commands:

network-clock-participate wic 0
network-clock-select 1 E1 0/0/0

The first line selects the WIC you wish to use, then the second selects the clock source (interface) you wish to use. Once I’d done that, the faxes magically all worked again.

http://www.cisco.com/cisco/web/support/index.html – TAC case collection requires a CCO login.

YouTube – Pigeon: Impossible.

AWESOME!

YouTube – simonscat’s Channel.

A really great little series of animations. Wonderful stuff!

YouTube – Eigenharp Roadshow – Air Studios.

Possibly one of the coolest instruments in a long time!

If you’ve ever worked on a Cisco IOS device, you’ve probably encountered the joy that is the terminal monitoring of events. IOS will happily dump bits of information onto the screen (very useful) but it’s also horribly disruptive if you’re typing in a command. Everything disappears into this sea of debug output or other info.

In my case, I often do ISDN Q.931 debugs to find out what’s going on on my voice gateways. I’ve gotten used to it now, but typing ‘undebug all’ whilst a ton of information whizzes by is less fun than it sounds.

Today, I found this useful command:

“logging synchronous”

If you enable that on your vtys or con (or aux) ports, it basically causes IOS to dump the debug still, but it’ll do it on all other lines EXCEPT the one you’re typing on, leaving it alone. I imagine it could cause some slowdowns as it has to draw your command in the same place, whilst interleaving other junk, so it could potentially cause problems on a direct console, but if you remotely telnet (I really should enable SSH one day), it’ll be a lot easier to see what’s going on.

Here’s the gist of what you need to do:

  Router(config)#line con 0
  Router(config-line)#logging synchronous
  Router(config-line)#line vty 0 4
  Router(config-line)#logging synchronous

Then wr mem as usual.

Easy!

It’s really rather simple:

Act dumb.

Yes really. Allow me to explain.

I browse a lot of forums, and in many cases I’ll see a barrage of people complaining about various technical support departments at ISPs, service providers, and whoever else. They’ll be wailing that they didn’t get the help they wanted, and nothing worked, and how much the support sucks, and so on.

I’ve called support for Virgin Media, Be, Zen, Orange, and various others over the years. I work in IT, and I do a lot of troubleshooting myself. I often have an idea of what the problem is, and sometimes what might need to be done to fix the issue. I could easily phone up the support line, tell them what the issue is, and demand they do what I want to fix it. I might be right – but there’s an equal chance I’m NOT right, and could send them on a wild goose chase trying to deploy the wrong fix. I don’t know their systems or their networks, so who am I to tell them what to do?

And that’s the issue most people have. They may, or may not know what needs to be done, but the service provider know their stuff better than you do, so let THEM decide what to do. Hence why I say ‘act dumb’. I had an issue with my cable modem. I phoned up, and just told them what they asked for. I did exactly what the guy on the other end told me to do. An engineer was dispatched, and the problem was fixed. Simple. Other people with similar issues (it was an attenuation issue) go through countless engineer calls, and visits, and personally I think that’s because they pre-empt things, and try and tell the support person what to do. I never do.

Yes, I could phone up and say ‘I work in IT, therefore I know what the issue is’. They probably get loads of calls like that every day. As I say, some will know what the issue is, others will only think they do.

I had issues with my ADSL flaking out (back when I was on ADSL), and I jumped through all the hoops both Zen and Be had, which included moving the router to the master socket, and running through all the tests they wanted to do. As a consequence of doing that, all the BT engineer visits I had, which led to a No Fault Found, weren’t charged to me. If you read all the paperwork, in theory I should have been charged about £75 for those visits, but I wasn’t as they were raised as per the processes BT set up for the ADSL providers. We jumped through all the right hoops, so I wasn’t liable for the costs. If I’d phoned up, told them I wasn’t going to move the router and demanded an engineer, I would have been stung. I’ve seen people do that, and wonder why they get charged.

Don’t get me wrong – there ARE plenty of ‘bad’ support lines out there, and poorly trained people manning them. They may be working to a script, but think of it this way; if you were in a play, and one person is working from a script, and you turn up and decide you don’t want to use the script, you’re going to very quickly end up in a mess. Same goes for these support lines really. Let them follow their script, and don’t pre-empt them. Let them ask the questions they have to ask (have you turned it off and on again?). If they can’t handle it, they’ll escalate it anyway. I’ve gotten in and out of support lines in no time, and had all my issues solved pretty quickly, without any shouting, swearing, or demanding to speak to their supervisor.

So remember. Just act dumb!

I’ve been wrestling with the Cisco Telephony Service Provider (TSP) today. We’re doing an install of ARC console, and having set everything else up, it wouldn’t start. I traced back and found that the issue lay in the TSP module itself. ARC tell you to try a utility called TAPI Soft Phone. Every time I ran it I got a vague error about not being able to connect.

I found the TSP was actually dropping logs into C:\Temp. I found these lines:

CiscoTSP001.tsp|   CSelsiusTSPWaveList::GetAvailWave() *ERROR* No wave available|<LVL::Error><MASK::0001>
CiscoTSP001.tsp|   CSelsiusTSPDevice::OpenDevice() [ARC-SrvQueue3] *ERROR* GetAvailWave() returned WAVELIST_NOT_ASSIGNED|<LVL::Error><MASK::0001>

I was aware there was a Wave driver in there, but it supposedly gets installed when the TSP installed.

Right?

Well, it would seem not. Whether it’s meant to or not, I don’t know. I could find plenty of reference to reinstalling it, but not a lot of mention of how to install it in the first place (or reinstall it for that matter). A bit more digging, and I found this Cisco document which tells you how to install.

It mentions Windows 2000, but it’s roughly the same for 2003, which I’m using. I selected Add New Device, Game and Audio controllers, and just pointed it at the driver.

One thing that struck me is that Windows claims the driver is unsigned. I’m wondering if the TSP installer is trying to shoe-horn the unsigned driver in, failing, and just silently giving up. Like I say, that’s assuming it actually does it in the first place.

Anyway it now works. A simple thing, but it took me a while to find the root cause and fix it.

This recipe is a sort of mish-mash of ideas from other recipes. Rocky Roads are actually pretty easy to make – not so much a recipe as a ‘collect some random stuff and mush it together with chocolate’.

Anyway, this is what I made this weekend:

• 50g Butter
• 40g White Chocolate Buttons
• 100g Raisins
• 100g Marshmallows (either small, or large ones cut down smaller – random sizes are more fun)
• 200g digestive biscuits
• 460g Cadbury Dairy Milk chocolate

1. Melt three quarters of the chocolate with the butter. Save the remaining chocolate until the end (don’t melt it yet). Be careful not to cook the chocolate.
2. Break the biscuits up into smaller pieces, and mix with the white chocolate buttons, raisins and marshmallows.
3. Pour the chocolate onto the dry ingredients, and mix it all until evenly coated.
4. Line a shallow tin with greaseproof paper, and place the mixture into it. Spread it out evenly
5. Melt the remaining chocolate and pour it on top of the main mixture
6. Set in a refrigerator for two to three hours
7. Cut into slices/squares/consume as one piece

That made a pretty nice mix. I think you could put in more fruit, and definitely more buttons. You could probably also get away without having the butter, but it helps to thin the chocolate a bit and make it spread and stick better.

The end result looks vaguely like this:

“Om nom nom” are the words you’re thinking of, I believe.

I might take another whack at it with more fruit. Anyway, enjoy!

I’ve been running Windows 7 on my main PC all this week, and I’ve very quickly found myself getting used to it. Even the new Start Bar is growing on me daily.

I decided today to see if my little HP 2133 netbook could take it. The HP has no optical drive, and I have no external USB drives. I had a IDE-USB converter thingy I’d used with an IDE CD drive in the past, but I’d blown the PSU on it. I needed another solution.

I’ve installed Linux from USB drives in the past, and I knew it was possible to install Windows onto HP Proliant servers using Smartstart and a USB drive. I was vaguely aware any PC could do it, you just needed a method.

A quick google turned up this blog entry:

HOWTO: Install Windows Vista from a high speed USB 2.0 Flash Drive – Windows Live.

I reasoned that Windows 7 is basically Vista so the theory was probably the same. I got my 4GB SanDisk USB drive, downloaded the Windows 7 Home Premium ISO from Technet, and started setting up the disk.

After plugging it into the netbook, I powered up, and selected it from the boot menu. Imagine my surprise when it worked! Not only that, but it booted, and installed perfectly fine. I wondered if it had done something stupid like installed the boot loader on the USB drive (had that happen before) or messed up the drive letters, but no, everything is where it should be.

What impressed me the most was that during the setup, it found my wireless controller, and set up my connection to the AP. Next it activated with Microsoft (although oddly told me it had expired, and must be activated), and I was in. Windows Update was already showing activity so I had a look and found all the other drivers waiting to be installed. Once they had installed and rebooted, I have a fully set up Netbook!

The start bar looks like it could be very useable on the netbook, and it seems pretty nippy. I’d run XP on it before as I decided Vista was a pointless excercise. Don’t get me wrong, I rather like Vista, and had been running it for some time, but it really doesn’t get on well with low-end hardware. This little machine seems to be OK with Windows 7 however, so I’ll have to see how it goes.

Not that you’d notice.

If you’re reading this, then you’re seeing the newly hosted site. Nothing has changed, I’ve just moved to a virtual private server, where I can host all my domains, and do all my own setup and control of the hosting.

I’ve had years of excellent hosting from UK Web Solutions Direct, so I was a little sad to leave them. But if you’re looking for a basic host, I can’t recommend them highly enough, so if you’re looking for a good host, give them a go!