RE: How Realistic is the Dom Perignon Test

Re: How Realistic is the Dom Perignon Test -- Part 1 -- Jean Ichbiah
Posted by John Harms , Wed, Nov 27, 2002, 17:28:58 Reply Top Forum

Jean suggested that I post this email discussion on the topic.

Let me start by saying that I think FitalyStamp is a great
product. If MessagEaseST hadn't come along, I would still
be using FS and recommending it to others. However, FS has
certain shortcomings -- as all products do. MessagEaseST
happened to solve the shortcomings that affected *me* --
hence my own switch. Other FS users who have not
experienced those problems will have no reason to switch.

The main topic of this email exchange between Jean and me
was the contents of the DP text. However, along the way
some other interesting side topics came up.

Key:
>>> My original comment
> Jean's response
My response to Jean's response

------------------------

>>> Thanks for running the fun DPIII contest. Originally, I
>>> assumed that FS would certainly win, with tap-tap-tap on
>>> text with all letters and only two punctuation marks.
>>> However, it looks like Keith ("MC Skoop") could give FS a
>>> serious challenge for the #1 spot!

>>> BTW, I respectfully disagree with Gordon Walker's conclusion
>>> in your recent forum posting, "How Realistic is the Dom
>>> Perignon Test -- Part 1." (He said, "The test seems
>>> eminently reasonable.") However, I also disagree with Arthur
>>> Hagen's assertion that the test is totally biased in favor
>>> of FS. Both positions are too extreme. Clearly a
>>> letters-only phrase measures *something* of value, since
>>> most text entry is indeed composed mostly of those letters.
>>> And, Walker is correct that there is no single PDA usage
>>> model.

> Why not repost this discussion as a response to the two
> messages as it will be of interest to others.

Done.

> I've read your comment on the ME board and responded to
> a similar comment in an exchange with Keith:

> By the way, I am reading some messages on the ME board
> and some of the comments are amusing -- saying that the test
> is biased in favor of Fitaly because there are no numbers.
> Phone numbers like (781) 272-3200 would be done on the
> 123 panel faster in Fitaly than on ME. Then if we really
> wanted to bias a test we would put some of these
> Canadian postal codes like R3C 4A5

Clearly, we are on different wavelengths here. My background
assertion is that numbers are easier and faster on
MessagEaseST than on Fitaly. Your assertion seems to be that
numbers are better on Fitaly, and thus numbers in the DP test
would bias it *toward* Fitaly. There is no way to resolve
this difference in assertions without data -- otherwise it's
just "my word against yours." The data could come from a
DP-like test that includes numbers.

> The point is that when you get to type numbers, they usually
> come with punctuation or degrees or $ and the 123 panel is
> then unbeatable. When you get just one or two, then sliding
> the punctuations is extremely fast...

My personal experience was that "normal" punctuation on FS
was cumbersome and error prone, with its need for shifting,
split cells and/or popup grids. Obviously not everyone has
this experience -- but for people like me, MessagEaseST's
always-available right-there punctuation is much faster and
more accurate.

> try the infamous Canadian postal codes on ME :-))

I have; they are easy. One of the reasons I switched from
FitalyStamp to MessagEaseST was the FS number panel. I
tried for about a year to get used to that FS
number/punctuation panel, and found it frustratingly slow
and error-prone, while my letter input was very fast.
MessagEaseST, on the other hand, has the numbers arranged in
the familiar phone arrangement, with keys big enough to hit
confidently in a hurry.

>>> However, having *only* letters (and two periods) in the test
>>> fails to accurately measure the everyday usage of PDAs --
>>> except for the very few professional writers who write
>>> novels using only a stylus.

> The truth is (1) that punctuation is very rare - confirmed
> by the Brown corpus - and

Sorry; I'm not familiar with that study.

> (2) that Fitaly is very fast on
> punctuation, How do you do an m-dash - in ME? :)

Easy: type m- (an "m" and a dash) and follow it with the
Combine stroke. Likewise, the other non-mainstream
characters have mnemonic ways to combine two characters into
the desired character. (Can you guess how ME does an
"n-dash"?) But in any case, both MessagEase and FS users
will assign favorite non-mainstream characters to
memory/slide locations.

>>> So, what's the solution? I propose using my 2.5 MB memo
>>> archive as datapoint to help us figure out where to go from
>>> here. I use my Palm instead of paper to take notes in
>>> meetings, brainstorm, record phone conversations, etc. I
>>> think that a year's worth of real-world note-taking would
>>> more accurately reflect the PDA-based distribution of
>>> letters, numbers, and punctuation than "Tale of Two Cities"
>>> does.

> We did our own statistics using more than 20 megabytes of text
> in various domains: scores of emails, messages in newsgroups,
> medical transcription reports. See also our other product
> line: Instant Text.

That's good, although those are not necessarily
representative of meeting-note-taking PDA input. However,
it is certainly better than the text of a novel.

>>> I don't know what the results will be. Perhaps the quantity
>>> of punctuation and numbers is actually less than I think.
>>> But I know it *feels* like there are many, since I found
>>> that aspect of FS to be very frustrating. (This was back
>>> when I was recommending FS to others because it was clearly
>>> the best entry tool at the time.) If the results show that,
>>> say, 1% of the text is numeric, then perhaps DP4's
>>> 181-character phrase should have 1 or 2 numbers. (And so
>>> forth.)

> We are then talking about something that is less than 0.5%

My own impression is that it is more than 0.5%, but only the
analysis will tell for sure. Stay tuned.

> Let's keep in mind that this is a fun contest with a very
> vague connection to statistical value. It shows with 99%
> certainty that handrwriting is not as good as the alternate
> methods and by a huge factor.

Indeed, there is no disputing that a tool like Fitaly is
vastly better than Graffiti, and even handwriting. The
contest makes this point clearly and in a fun way.

> I think that Fitaly will win
> because spaces take too long a move in ME...

Ah, but we are differing in background assertions again. My
assertion is that spaces are faster and less error prone in
MessagEaseST because the spacebar is a much bigger target.
This is based, not on mathematics, but on personal
experience. My most frequent error in FS was missing the
spacebar and getting a letter instead.

> ...and having to
> slide every second or third letter will put ME at a
> disadvantage.

On the other hand, those slides are confined to a relatively
small square. FS forces the user to dance all around the
digitizer.

> On the other hand, the results on the contest will allow
> ME supporters to say that theirs is a good method.

Indeed. Whoever *wins* doesn't necessarily mean that the
method used is therefore the best overall solution. It
could just mean that the virtuoso pianist happened to do
his/her performance on a particular brand of piano rather
than another. The DP contest is a fun vehicle for certain
people with exceptionally fine motor/memory skills to shine.
"Ordinary" users of either method won't be able to aspire to
winning times for day-to-day use of their PDA.

> Note also that Fitaly users enter freely - no cherry-picking
> - and there are slow entries as well as very fast ones,
> which makes the average more meaningful. (I am getting the
> impression that Saied Nesbat is understanding the value of
> having a large sample, witness his invitation to
> participate, yesterday on the ME forum.)

Indeed, it is taking awhile for the MessagEase users to get
excited about the DP contest. I suspect it's just not yet a
tradition like it is with FS users.

>>> If nothing else, that would silence the naysayers about the
>>> test's validity next time.

Another way to look at it is: "What could it hurt?" If the
DP-4 contest had text with numbers and punctuation
proportional to real *PDA* data, no one would be tempted to
claim that the text was not realistic.

>>> Would you be interested in the results of my memo data
>>> analysis?

> Definitely.

Will do; stay tuned.

As I said above, that will become a data point to continue
the discussion. If the number turns out to be less than
0.5%, then we don't need to go further. If the number is
higher, perhaps we could analyze other users' data as well,
to get a more comprehensive data set. PDA program idea: go
through *all* non-encrypted data in the standard PDA
databases and get character frequently stats, written to a
Memo. Users could run the program and post the results.
Just a thought.

-- John :-)
_________________________________________________________
|
| John Harms (photojhh@yahoo.com)
|_________________________________________________________


Author | Edit | Reply Original Top Current page
Followups