Announcing the SpeakRight Framework

There's been a big change in direction for me in the last month. I'm going open-source. Spent the last month learning Java and Eclipse...and creating an open-source VoiceXML framework called the SpeakRight Framework. It's a code-based approach to writing speech recognition apps that, I believe, results in much higher code re-use and therefore faster development time. Also, modern Java IDEs are amazingly smart and helpful. A good Intellisense (called Code Assist in Eclipse) is as good as a GUI-based/property page approach.

Currently SpeakRight has only been tested on the Voxeo community site. It's VoiceXML 2.0.

A key part of the plan are "SROs". SpeakRight Objects are re-usable speech objects that, being open-source, should help encourage a growing body of components to build speech apps out of.

Can't wait to try SpeakRight out on Linux, and on MSS 2007!

Grokking StringTemplate

The StringTemplate template engine is a powerful tool for generating markup text or code. It does, though, take a little getting used to. The templates may like a type of programming language, but there are some key differences. And since it's not compiled, bad syntax may just fail silently. Luckily it's quick to try things.

Update: I have corrected errors in the original post. Namely $if(it.CountIsOne)$ is the syntax for referencing $it$ inside an $if$.

First of all let's start with something that works. We want a field tag to output a prompt tag for each prompt in the list promptL Assume promptL contains an ArrayList of two strings "a" and "b". This is the XML we want (ignoring whitespace issues)

<field>
<prompt>some a</prompt>
<prompt>some b</prompt>
</field>

Here are the templates that successfully did this.

//generate the field tag
field(promptL) ::= << <field> $prompts(promptL)$ </field> >>

//generate a tag for each item in the list. There are other prompt tags so we parameterize tagname
prompts(promptL) ::= << $promptL:prompt(tagname="prompt"); separator="\n"$ >>

//output a single tag
prompt(tagname) ::= << <$tagname$>some $it$</tagname>
>>

Now let's add one thing to the output and see what template changes are required. Let's say we want each prompt tag to have a count attribute that is "1" for the first tag, "2" for the second and so on. Like this:javascript:void(0)
Publish

<field>
<prompt count="1">some a</prompt>
<prompt count="2">some b</prompt>
</field>

This is easy. We can just use the built-in $i$ to do this:

prompt(tagname) ::= << <$tagname$ count="$i$">some $it$</tagname>
>>

Now let's say we want to output the count attribute only when the count is greater than "1". This is where things got tricky.

This DOESN'T WORK. ST doesn't let you compare values in $if$. This would go against the spirit of ST whose goal is a strict seperation between presentation and model logic.

prompt(tagname) ::= << $if($it.count$=="1")$ <$tagname$ count="$i$">some $it$</tagname>
$else$
<$tagname$ count="$i$">some $it$</tagname>
$endif$
>>

Maybe we can replace the strings in promptL with Java objects that have a getCountIsOne method. We'll also need a getText method to get the string as well. Recall that $it.someFunc$ resolves in Java to a search for a method named getSomeFunc().

This WORKS!. Just remember the syntax is is to leave the '$' characters off it when inside a $if$. You're inside $$ already.

prompt(tagname) ::= << $if(it.countIsOne)$ <$tagname$>some $it.text$</tagname>
$else$
<$tagname$ count="$i$">some $it.text$</tagname>
$endif$
>>

Another approach is template application. We change our Java method to getCountNotOne and have it return null when it's the first item in the list, and the count otherwise.

This WORKS! We apply the template docountattr on $it$

prompt(tagname) ::= << <$tagname$ $it.countNotOne:docountattr()$>$it.text()$</tagname>
>>
//a way of outputting count attr only when count not 1
docountattr() ::= << count="$it$">>

Template application is powerful. You can pass parameters in as well.

THIS WORKS!. We're applying the template prompt with two parameters tagname and bargeIn.

//generate a tag for each item in the list. There are other prompt tags so we parameterize tagname
prompts(promptL) ::= << $promptL:prompt(tagname="prompt",bargeIn="true"); separator="\n"$ >>

prompt(tagname, index) ::= << $if(it.CountIsOne)$ <$tagname$ count="$i$" bargeIn="$bargeIn$">some $it$</tagname>
$else$
<$tagname$ count="$i$"bargeIn="$bargeIn$">some $it$</tagname>
$endif$
>>

Once you've grokked things, StringTemplate gives you excellent separation of presentation from logic, and is very good at extracting what it needs from ordinary Java objects.

Learning Java

After years in the Microsoft I am finally learning Java and Eclipse. For anyone familiar with C#, Java is extremely close but feels its age. No properties, delegates, or events? I can live without them. Eclipse however is another story. What AN AMAZING PIECE OF SOFTWARE! Miles ahead of Visual Studio in many areas. Mainly it just works; which is a huge kudo to any piece of complicated software.

And Eclipse is free. In fact, one person with a laptop, GMail, Eclipse, and SourceForge is the equivalent of a five person company from the 1980s.

Open-source software is unstoppable in many ways. You just can't compete with free. Software tool vendors have to keep moving into more and more niche areas. All the big ones, like source controls, IDEs, and bug tracking, are well covered by OSS. That being said, Microsoft has something like 50 billion dollars in the bank. They could not earn a cent for the next ten years and keep paying every employee. That's kind of unstoppable too!

Daily backup of a laptop

Thanks to Gerald Gibson Jr.'s great article I now have a free way to do daily backups of my laptop. Windows natively supports the zip format (as "compressed folders"). The article is a C# program that generates ZIP files using the Shell API.

(Be sure to get the latest version from Gerald's web site)

My app zips several important directories into zip files in c:\zip. Then, using a free account at box.net I upload the zip files daily. The free account has a limit of 1 GB and each file must be less than 10 MB. But for 5$ a month you get much larger maximums.

Not the finest solution; but it'll do for now. KISS.

Update: A Mozilla plug-in called GSpace turns your GMail account into a FTP site. Each file you upload becomes an e-mail. GMail now has 2.5 GB of storage.

MySpace is Show Biz

Great article by David F. Carr on the behind the scenes mayhem at MySpace, whose exponential growth from a few hundred thousand users to 30 million users overloaded the system.

"MySpace started small, with two Web servers talking to a single database server. Originally, they were 2-processor Dell servers loaded with 4 gigabytes of memory"

Again and again MySpace was re-written to add: replicated databases, SANs, ASP.Net, and 64-bit servers. One interesting lesson was the addition of caching

To further lighten the burden on its storage systems when it reached 17 million accounts, in the spring of 2005 MySpace added a caching tier—a layer of servers placed between the Web servers and the database servers whose sole job was to capture copies of frequently accessed data objects in memory and serve them to the Web application without the need for a database lookup.

This sounds a lot like what GigaSpaces is offering. It's a LindaSpace-like type of middleware that replaces a messaging paradigm with a associative shared memory approach. Intense caching and heuristics to move data closer to its clients.

Scott Rosenberg makes the point that Web 2.0 successes may look easy from the outside, but behind the scenes there are huge challenges. Like Show Biz!

Why to use code generation techniques

Mark Baker comments on the problems with (xml) document validation. Any DTD that is too detailed is a time-limited contract. ProductType may be "1", "2", or "3" today. But a year from now types "4" and "5" may be allowed.

True, but validating, or what used to be called "laundering your input", still has to be done. Whether it's in the gatekeeper or the business object, the code has to be there.

But that’s really no way to write long-lived software, is it?

Yes, it's fine actually. Who cares where the error message "producttype '4' not allowed" comes from. All that matters is that at any instant in time it either processes valid messages or returns a suitable error message.

If splitting validation code from the business logic represents a maintainability problem, then consider code generation. Frameworks like StringTemplate make code generation easy. CodeGen tackles the common problem that a feature requires changes up and down the software stack. The traditional way to do this was manually make all the changes and rely on comments and checklists to ensure all the required changes are done. With CodeGen you can use a more literate programming style. Define in one place, generate various bits of code, and inject it into the appropriate places in your code. Patterns such as Chain Of Responsibility help.

The Beauty of Unreal

Further to my recent post on example-centric programming, Tony responded with

I entirely agree with the audience targeting critique you make. I was thinking about the historical roots of programming, the social aspects, and how they must be considered, not just for fuzzy artsy reasons but for because they are part of the sociology of skills and attitudes that mold the field (we are the medium of programming).

I think programming has strong roots in telephony and in code-breaking and
in mathematics. These roots influence the need for interfacing at the hardware level,
efficiency and high abstraction.

The PC revolution is in a way an extension of the business view of data processing, which was very much separate from the scientific programming done on mini-computers. In fact now that I think of it, this may be why DEC missed the boat on PC's - PC's are for business and now are appliances, not really computers - but data processors (ironic given the PDP name - there's a story there...).

So what this guy is advocating is a reconciliation of these two worlds, a wrapper view. Not sure that can fly in social terms, for the reasons you identify, and because we have two worlds in software, the programming in the old telephony-coding=math sense and the data processing view with forms, HTML, windows and high-level object libraries.

Alan Kay's deep thoughts on similar topics are here:

I hadn't really thought about the two ends of programming like that before. I guess there are two cultures. Part of my confusion is from watching his SubText video, where he shows how it works. Although it meets all the aims in the manifesto (text-is-a-dead-end, don't-copy-and-paste, avoid-control-flow), the feel of video is very mathematical. It feels like LISP programming with recursion. Recursion is a strange concept to most people; like postfix notation. In this way SubText (as it exists in the video) is a failure for end-user programming. Ordinary users will prefer 10 PRINT "HELLO" style of programming.
Perhaps he can take SubText into more of a wrapper thing that works at the component level. Who was that guy in the 1980s -- Brad Cox, who talked about software ICs. Connect up the output of this algortithm to that report view...

Speaking of "Language Matters" and "It's usability, stupid", have a look at the Unreal Tournament Engine. It's become that gaming engine of choice for lots of top games. A very clean design with a virtual machine and a simple language, but extended with a few key gaming features, like states.

Unreal Engine architecture

Example-centric programming

This Jonathan Edwards guy is brillant. His demo of Subtext is the most creative thing in programming in years. But he's basically trying to replace 50 years of programming culture: programs as text. Doomed to fail, like Charles Simonyi's Intentional programming, but may throw up interesting mashups.

End-user progamming is a dubious thing. It comes and goes as the Next Big Thing. The problem is that most people don't have the patience to write & debug software, no matter how wonderful the syntax is. Even something simple (to programmers), like control flow is a powerful and therefore dangerous notion. If you remove you drastically limit the expressiveness of the language; if you leave it in, people will stumble into the pitfalls.

End-user content, though is a killer app. My 10 year old son creates animations, web sites, and videos. A big part of video games now is the design-your-own-superhero. Choose body type, hair colour, clothes, etc. They often spend longer on it than playing the game. This isn't programming in a traditional sense. It's a type of visual programming where users assemble pre-existing components that know how to fit together. The components do the heavy lifting, while the user still feels they are "telling the computer what to do". And isn't that what programming is all about. Drag-and-drop telephony toolkits for IVRs are much the same :)

When more is less

In retooling our QA lab we've dropped the number of machines by half...and become more productive! Our work involves constantly re-installing windows service packes, hardware drivers, and assorted telephony software. Turns out that looking after of lots of machines was wasting time. We didn't used to have an organized way of managing the machines; we didn't know what was already on them, especially after a week of testing.

The cry always went out for more machines, which arrived "clean" and ready to go. Of course, after a testing cycle the new machine could end up just as messed up as all the others. Later, when the next release rolled around it was "we need more machines" all over again.

Reminds me of the old tale about the workbench with 25 screwdrivers and a set of storage slots for the screwdrivers, but only 24 slots. In the beginning each screwdriver was in its appropriately marked slot. Except #25 which someone borrows. Then another worker borrows #15. When the first worker returns, he puts #25 into the only open slot, #15's slot. Then he borrows #7. Now the second worker returns and puts #15 into slot #7. Pretty soon all the screwdrivers are in the wrong slots. The whole storage system fell apart because of a single insufficient slot.

Even in Windows-land, OSS is here

We installed Subversion and Trac here last week to replace Starteam for defect tracking and source control. Firefox and GMail/Yahoo/Hotmail are rampant. Asterisk answers the phone. All these open-source software tools have replaced commercial products. Even here in a traditionally Microsoft shop, OSS is starting to mainstream. And we didn't need to switch to Linux to do it. Yes, the Asterisk box runs Linux, but it's just a box on the network, like GMail is a service on the larger network called the Internet. Who cares what GMail servers run; Windows clients can use it happily.

Yes we still use Exchange, Visual Studio, and GotoMeeting.com. But in five years?

Ballmer on Vista

Scott Rosenberg has an interesting article on Ballmer's comments that Vista has come out pretty much as expected. I think there are about 30,000 programmers at the Redmond campus. It boggles the mind how that many coders can get anything done that doesn't duplicate, simulate, or break what others are doing. For many years they coped through specialization. Each programmer got something tiny, like a single Win32 API call, to work on.

Then after XP they pinned the needle the other way, and a rewrite orgy began. It produced some good things. XAML is good. They completely rewrote Window's TCP/IP stack; which is quite an act of chutzpah, and now it supports multi-core CPUs in useful ways. But overall they ended up suffering exactly the sort of problems that a Big Bang approach would bring.

In a way it's a smart move though. Most people are quite happy with XP. Any next version had to be a long-term, next-generation thing. People will love Vista in 2009.

Estimating Software Projects

Estimating how long it will take to finish a software project is one of the toughest parts of software engineering. Luckily it's easier to get better: start making estimates now and track your accuracy.

Two rules of thumb:

1. Make a guess and double it. I find this one surprisingly accurate. Probably because I tend to work on the same sorts of projects; but then don't most programmers!
   
2. Brook's book The Mythical Man Month says that every app has a core set of functionality that can be done in X weeks. To create a product will take 9X because of all the wrapper code for config/reporting/error handling, and all the user documentation that needs to be provided. People constantly underestimate this factor of 9.

We normally provide estimates that include design, coding, user docs, but not QA. My manager likes an estimate to broken down into a list of tasks that fit on a single page. Tasks that you are most unsure of should be clearly identified by using a range such as "4-10 weeks".

The whole game of making & meeting estimates is about what features are "in" and what's "out". I like to tie estimates to an explicit list of what features users will and won't get. Requirements are bound to change, and managers sometimes have a habit of forgetting that your estimate was tied to a feature set.

As usual, Joel has written on this already.

You can't tune silence

Tuning refers to the post-deployment tweaking of grammars in a speech app. Your QA department can test the features, but there's nothing like real callers in all their variations of speaking styles to really probe your app's grammars. If the open-source mantra is "with enough eyeballs all bugs are shallow", then the speech app mantra should be with enough speakers all grammar bugs are shallow.

Tuning is time-consuming because a human has to listen to the audio recordings of hundreds of phone calls. If the system can log speech rec errors, then the person doing the tuning can zero in on those utterances. If not, and we will see how this can happen, tuning becomes very difficult.

Out-of-grammar errors are the easiest to detect because the speech recognizer returns a NoRec error. These may be due to coughs, background noise or other audio problems that grammar changes can't fix. Others are due to variations in pronounciation that your grammar doesn't know about. People's names are notoriously hard this way because: (a) names are multi-lingual in origin, and (b) many people will mis-pronounce a name they've only read. We had a Mr Biber here, pronounced bee-ber, but many people would say bye-ber. Tuning revealed this, as well as the fact that some people would say only the last name. The grammar needed to be changed to cover all these possibilites.

The more difficult problem is what we call the Rumplestiltskin error. Most speech engines want to recognize. Try saying "Rumplestiltskin" to a speech-rec auto-attendant. You'll be suprised at how often it will find a (wrong) match. Saying something completely outside the grammar will cause a medium-confidence recognition of some wrong word in the grammar. Of course you can affect this by raising the recognition and confidence thresholds, but that may cause other unwanted recognition problems. Confirmation, for example, is an added step in the dialog that callers can grow to resent. The problem with a Rumplestiltskin error is that it's invisible; the system really has no knowledge that a recognition error has ocurred. This makes finding the error a time-consuming task for the tuner. The caller may be saying an acceptable alternative for a word, but until we discover this and add it to the grammar, the system will not be pleasing this caller.

Tuning becomes a big problem when dyamic grammars are used, because you can't do any pre-deployment tuning. The app will read a list of phrases from a database, say Little League team names, and generate a grammar. Consider the "Nowell Gnats". It's not unusual for the TTS engine to pronounce a word differently from what the speech rec engine accepts as a pronounciation. This is bad because your prompt may say it as "you can say Nole Nats..." whereas the speech rec engine wants "Now-well Nats".

It's easy to end up with a situation where the app won't accept any sensible pronounciation for a word, and even tells callers the wrong pronounciation to use! All without generating any recognition errors. Silent failure.

Can the speech rec platform help? Yes, here are a couple of ideas. First, the app should have "tuning mode" that can be set temporarily. It increases reco thresholds to force more confirmations, and logs all rejected confirmations as candidates for tuning. Secondly, the system should have a batch process that studies patterns of calls. If the same person calls back in several times in a row and never seems to complete a transaction, then these calls are also tuning candidates.

Bye Bye Speech Server -- Hello SPS

So, Microsoft Speech Server is being folded into Office Communication Server, and will be known as Speech Platform Services. This was announced at today's SpeechTek conference. Office Communication Server (OCS) is basically Live Communication Server, which does presence and SIP call routing.

I have fairly mixed views about this move. First, the negatives. Yet again, Microsoft is making a mid-course correction with their speech offering. At the beginning the vision was multimodal speech. Then it was telephony-based speech rec by web developers (by sprinkling "SALT" on their web pages to speech-enable them). Then it telephony-based speech rec using more standard methods, such as VoiceXML. Now it's speech rec as part of an enterprise information system. For developers actually trying to build applications, all these course corrections are unnerving, to say the least. If the overall goal, as Microsoft says, is to create an "ecosystem" then they need to realize that stability is a key factor. Wandering climate change isn't helping.

Now the positives. OCS is a strategic product for Microsoft. Communication is becoming a key part of every organization. Communication across devices. Synchronous and ayschronous communication. Features such as: presence, IM, VoIP, video, Find-Me, Follow-Me, and ad-hoc conferencing. Think EMail++. For speech rec to be bundled into such a strategic product is a huge win. It also shows that the speech folks at Microsoft have been listening and learning. Changing course often works out better than not changing course!

Lets hope the transition to OCS is as seamless as possible for existing MSS speech developers.

Nice Clutch Play by Microsoft

Anyone who does speech recognition demos knows how they can go bad. This one by Microsoft's Rob Chambers (YouTube) is about as bad as it could get. After mis-recognizing the presenter's first utterance, the recognizer got even worse at understanding his correction attempts like "select all and delete".

Turns out it was a Longhorn bug with its audio software. Today Rob did the demo again, at SpeechTek no less! Eight minutes long and it worked fabulously. It was a nice comeback and a gutsy move -- way to go guys!

Charlie Wilson's War -- Book Review

The sub-title says it all "The extra-ordinary story of the largest covert operation in history". George Crile has written a fascinating account of the CIA's role in the Afghan resistance to the Soviet Invasion of Afghanistan.

Charlie Wilson was a Dem senator from Texas. Most of the book deals with his efforts in Washington to get funding and support to his beloved mujahadeen. Washington is revealed not as a massive beurocratic system, or machine; but as a personality driven clique. One person, with the right skills (and Mr Wilson seems to have been a master), can find and grab the levers of power over literally billions of dollars in funding.

The other feature is the Alice-in-Wonderland nature of the world of the 1980s vs today's post-911 world. Then it was the Soviets who invaded a muslim country and faced a fierce insurgency. The US supported the insurgency at the urging of a Democratic president.
They supported (and were cheered) by fundamentalist jihadis, giving them secret weapons for attacking convoys and shelling bases.

One interesting question regarding today's Iraq is the lack of a Stinger missile. It made a huge difference in the Afghan war; as it would in Iraq. Strange that twenty years on there aren't tons of Stingers and its imitations available on the black market.

A movie is being made of the book.

C# Cookbook -- Book Review

If you program in C# then buy this book. It's an indispensible reference with 300 code snippets of everything you never have time to look up. Snippets on things like: better ways to use collections, regular expressions, generics, delegates, exceptions, reflection, i/o, XML, and security.

MSDN is great if you already know the name of the class or function that you're interested in. But if you don't, then it can be very hard. Date formatting, for example, is under string.Format() instead of under DateTime.

C# Cookbook is goal-driven. Each topic, such as "Writing a TCP Server", "Increasing StringBuilder Performance" or "Creating a Priority Queue" is a full solution to a programming problem. It covers C# 2.0 so generics and anonymous methods are covered.

Put it beside your desk. Read ten snippets a day, and in a month you'll be a better programmer.

Eating with Kids at the Bar-B-Barn

Eating with kids at restaurants can be a challenge, but last weekend in Montreal we hit a new high (or low). The Bar-B-Barn is a classic Quebec chicken & ribs place that takes its tradition seriously. With puritanical zeal, they have kept the same menu and decor: department of highways road-stripe-yellow paint, pine panelling, and ancient posters of hockey players.

The kids couldn't believe we were planning to eat in this low-ceilinged room with creaky floors and exposed heating ducts. My older son couldn't get over the menu. He kept asking "But there's only two items on the menu?". Yes, I'ld reply: chicken or ribs. "Just two things!?" and "Is that really it?" For someone used to the multi-page menus of a typically eatery, it blew his mind.

Son #2 is precise and his criticisms were detailed: the ribs had too much sauce, the fries were too long, and "this bun sucks!". Summing up, he concluded that "at least a crappy restaurant could have better lighting". Needless to say this sort of side commentary dulls the enjoyment of the others at the table. But we all got through it, and the food is actually pretty good. A broadening experience; for when I asked days later if they had told their friends about the Bar-B-Barn, they said "it wasn't that bad."

The Joy of Ethereal

Ethereal really saved our bacon recently on an MSS + SIP project. Calls from one gateway worked, but another gateway in California wouldn't work. Turned out that the RTP packet length was 20 msec on the "good" gateway, and 10 msec on the "bad" one.

Whoever thought up the idea of human-readable message protocols should get a Nobel Prize. Protocols like SIP, HTTP and RSS are so much easier to troubleshoot because of this. The bad old days of RPC, ASN.1, and ISDN are hopefully nearly gone.

Of course, security may force encryption of the human-readable protocols : so we may end up back where we started...

Memory leaks in .Net

.Net was supposed to make memory leaks a thing of the past. Turns out they can happen in C#, even pure .Net code that has no COM objects or unmanaged codes.

The culprit is async sockets.

The the buffer we pass to Socket.BeginReceive gets "pinned" in memory, preventing GC memory compaction. Since all new allocation uses the top of the stack, .Net has no choice but to keep allocating more Private Bytes.

Here's a thread about someone else having this problem

http://groups.google.ca/group/microsoft.public.dotnet.framework/brows...

Strangely, the addition of a Thread.Sleep(1) seems to make the memory
leak go away.

Basically, here is the BeginReceive callback function:

void AsyncReceiveCallback(IAsyncResult result)
{
VeoAsyncIOInfo thisConnection = (VeoAsyncIOInfo)result.AsyncState;
int num_read = thisConnection.m_socket.EndReceive(result);
if (0 != num_read) {
ProcessNewData(thisConnection.m_buffer, num_read,
thisConnection.m_socket);
}

Thread.Sleep(1); //hack hack hack

thisConnection.m_socket.BeginReceive(thisConnection.m_buffer, 0,
MAXMSGSIZE, SocketFlags.None, new AsyncCallback(AsyncReceiveCallback),
thisConnection);
}