In my experience on the Treo thumboard I think it makes a considerable difference. There isn't much of a penalty when you type the wrong key on the thumboard. When I hear the beep I just roll my thumb one key over to hit the correct key. Sometimes in my haste i'll press the correct key but also an incorrect adjacent key at the same time, and it makes an error beep but there's no time penalty. On my 70 wpm entry I made atleast 5 errors.
I think to get an accurate score from thumboard users is to make it so we have to backspace and retype the correct key or it won't let us move to the next word. I think I would only get 50-60 wpm on a test like this.