Decoding
This is a statement from K1JT about the integrity of WSJT decodes:
1. Do not check "Aggressive decoding".
2. Set the "Sync" threshold higher.
3. Delete or rename the CALL3.TXT file.
4. Pay attention to your interference environment and to the many
forms of numerical and graphical information provided to the
operator, and get involved in the decision process yourself.
5. Never believe a questionable decoded callsign until you have seen it twice.
Item #5 is absolutely foolproof. Mnemonic: "If in doubt, throw it
out. See it twice? Then it's nice."
I did some testing of JT65 with WSJT 4.9.7 after Joe made these statements..
First line is decoding the OH7PI file that is distributed with WSJT using K1RQG as the home call.
The second and third line is the very same file decoded again, this time the home call in WSJT is changed to G4RGK...
104600 8 -20 3.3 42 2 * K1JT OH7PI KP32
104600 8 -20 3.3 43 2 * G4RGK OH2BAP KP20
104600 8 -20 3.3 43 2 * G4RGK OH2BAP KP20
Same file, distributed with the program, different decodes. K1JT tell us that this paragraph above has no relevance because it is referring to an old outdated version. The version 4.9.7 is however presently used by a number of people around the world, including a recent DX-expedition, and most people are unaware of the existing functionality. After some of us pointed it out, Joe Taylor submitted this interesting information to G4RGK to put on his website:
"WSJT 4.9.7 had a coding error. In a very limited set of circumstances, the error could cause an internally correct decoding of a transmission around -23 dB to be suppressed and "overruled" by a low-confidence result from the deep search decoder. The user would see only the incorrect result. Everyone who used WSJT 4.9.7 had this same problem, but different entries as "My Call" would generate false decodes with different received data files. Your call and some others would make it occur with the OH7PI file. My call would generate errors with different wave files. Using WSJT 4.9.7, any call sign will generate these low-confidence false decodes occasionally."
Have a look at a few different calls decoded from the supplied test file.
Do not use 4.9.7 !
The file "call3.txt" injecting known data to the decoding process
K1JT states the following:
do not maintain the list. It is not a part of WSJT. A copy is
distributed with the WSJT installation package simply as a
convenience for new users. Nearly every active WSJT user maintains
his or her own version of the list."
Earlier version of WSJT crashes if you don't have a CALL3.TXT file. The program depends on it being there. Default install will place a CALL3.TXT file on in the WSJT folder, and the program will use it.
Checking the version 5.9.3
I did some testing with the late version WSJT 5.9.3 and one of the sample .wav files supplied on the download page. The install was exactly as the Install Wizard suggested, and to be sure I had no settings wrong I deleted the WSJT.INI file (as suggested by K1JT) and restarted the program with default settings. I found some interesting results. I have been accused of "mis-using" the software to get these decodes, but you can rest assured I have made an ordinary install, and as mentioned above used default settings as any user would do running the program for the first time. What is shown here, regardless of settings, is that the program is using known data in the process of decoding, why else would changes in a text file show up as decoded text.
163600 2 -26 2.5 221 4 # K1JT IK1UWL JN33 OOO
163600 2 -26 2.5 221 4 # K1JT IK1UWL Jn33 OOO
163600 2 -26 2.5 221 4 # K1JT IK1UWl Jn33 OOO
163600 2 -26 2.5 221 4 # K1JT IK1UWLE Jn33 OOO
"Hit F2 to open the Setup -> Options screen and enter your own call (or some
other call) in place of K1JT in the "My Call" box.
Then dismiss the Options screen and try to decode the SP6GWB signal again. You
will surely fail, because for this message successful copy was obtained as a
result from the Deep Search decoder.
(Callsigns in the CALL3.TXT database are paired up with CQ and with the
"MyCall" entry, encoded, and compared with the received information. If the
signals match to within a specified minimum confidence level, the matching
message is displayed.) If no good match is found, there is a small chance that
you will see a low-confidence false decode. It will usually be flagged with a
"?" mark to remind you to exercise your own judgment, based on signal
strength, sync level, displayed graphical information and your accumulated JT65
experience, as to whether the message can be accepted as valid."
Well, following exactly this tutorial, using version 5.9.3, SP6GWB can be
SP6GWBE, SP6GWBERT or anything I put in the CALL3.TXT file. Full confidence
decode, no flags attached.
(first line is the decode without changing the CALL3.TXT information, 2nd and
3rd are decoded after I altered the SP6GWB entry in the file)
002400 3 -23 2.4 245 3 * k1jt SP6GWB JO80
002400 3 -23 2.4 245 3 * k1jt SP6GWBE JO80
002400 3 -23 2.4 245 3 * k1jt SP6GWBERT JO8
And, changing any of the known information as locator or home call produces a "no decode".. It's all about the "specified minimum confidence level" that the decoder is using, and it is unfortunately not set high enough to produce 100% reliable decodes. There is however no way for the operator to know this.
Conclusion:
It is obvious
that the program is using the information in the CALL3.TXT file or the information in the "My Call" or "To Radio" boxes to help reproduce data that was not received. This data was not received over the air, and the level of injected data to enable "recovery" is only known to K1JT.As you will see below, a statement from K1JT tells us that JT65 is only capable of decoding unknown text to a level of -20db S/N. Again, this is basically the same S/N limit as a good EME operator needs to copy weak CW.
Now then, we have to ask ourselves a few relevant questions:
The info above raises the questions if JT65 is designed to perform better by giving up the need for receiving all needed information via the airwaves.
My tests have showed that there is no way to decode the sample file unless all the information is available on the computer. This would also be the reason for why the people who are using the program hang out on loggers, it is a good way to present the program with all needed information and put it in the computer RAM.
People then argue with me that random QSO's are made all the time with JT65. Well read on and I will show you why that is. It has to do with signal levels, and when they are good enough random works. Just like with CW. But as soon as the JT65 "deep search decoder" is being used known information is injected to the decoding process, else the program fails. During these contacts, unfortunately, the decoder has been given all information to "reproduce" the missing bits not received and the "truly remarkable EME QSO's" are not so remarkable.
At the bottom of this page you will find another example of how people have to inject known data when monitoring JT65B beacons to make use of the "deep search decoder". This time it is Bernd, DF2ZC who is handing out instructions. To me it looks rather obvious that by following the advice, presenting known information in the right places (i e MyCall and call3.txt), very little information is required to be picked up via the radio path. I do not understand why monitoring beacons this way is interesting, because it indicates very little about factual radio conditions. Fragments received are presented as full, genuine, decodes.
Extract from " Minimal QSO's and their validity" by DJ5HG http://dubus.ns.km1708.keymachine.de/DJ5HG.pdf Chapter 15 " At the JT65 Deep Search only the equivalent of two characters from the call sign or locator is necessary for decoding"
In effect, what he is saying in his Dubus article is that only two characters need to be decoded to print full EME messages containing more characters, for example
"DJ5HG SM2CEW KP15 OOO"
It should be noted that DJ5HG says two characters from the call OR the locator (which is also a part of the database). So linking a callsign to your own call might even have been done by using the locator information in the database. If so, no information necessary for a true EME contact has been passed via the airwaves.
In JT65 it appears that receiving "EW" is enough to say both calls, locator and report have been received.
To clarify the requirements:
Extract from EME operating guide for 432 and above that can be found at http://www.nitehawk.com/rasmit/g3sek_op_proc.pdf "Minimum QSO: The definition of a minimum valid QSO is that both stations have copied all of the following: 1. Both callsigns from the other station"
As an example of how JT65 is presenting information only when it is know, take a look at the following information from K7XQ. It is an interesting report he published on Moon-Net, that in my eyes show us that JT65 is not capable of copying full calls at low signal levels, an therefore contacts made using the DS decoder should be treated as invalid.
"Last week I completed my small 2 x 7 element 6 meter EME station. Before I worried about the transmit side, I wanted to test the receive sidesince this half of the station is complete. Acouple of nights ago, I saw that K7AD was on and willing to transmit via EME some test signals to see if I can receive anything off the moon on my end. He began by calling me, "K7XQ K7AD ". I was in monitor only mode and DID NOT enter his callsign and/or grid in the call boxes. I assumed that any signal I receive would be "random" and I would be able to copy him no matter what text he sent. His transmission continued for 30 minutes and I received over 9 transmissions of syncs between -30 and -22 db but absolutely no text was decoded. Just about the time that I told him that I didn't copy anything from him, I proceeded to enter his callsign and grid in the box and from that point on, I began getting full text decoded on the screen. I then went back to retrieve the wavefiles that contained only syncs and LO and BEHOLD, I got full text decodes from those wavefile sequences. The question is how do I know if the program copies only partial calls and then the program proceeds to " fills in the missing blanks " based on a lookup table? The other question I have is since the program knows who I am, how can I prove that the program needs to copy my calls from the station calling me to display full calls on the screen ???"
Good questions from Jeff K7XQ that surely more people in the EME community would like to see answered.
In the February 2005 edition of the 432 and above EME Newsletter from K2UYH K1JT states the following regarding JT65 and it's users:
"To maintain consistency with other EME practice, these hardy souls use exactly the same long-established criteria for what constitutes a valid QSO. "
Of course, this is not true by any means, and K1JT is making his own interpretation of how QSO's are to be made, effectively trying to justify his invalid protocol.
Conclusion: It is obvious to me that decoding two characters and then adding known information via the local computer database is not in any way compatible with present EME QSO requirements.
I would not hesitate to say that QSO’s made with the Deep Seach (DS) module in JT65 are invalid with respect to any existing QSO requirement.
It is therefore sad to note that recent reports of completing DXCC on VHF/UHF depend on a communication protocol that doesn't live up to the minimum requirement for EME QSO's, or any amateur QSO's for that matter.
Shorthand messages
K1JT states the following:
"
The decoder knows what the shorthand message structure is supposed tobe -- tone separation and alternation period -- and it looks for
something that matches one of these."
This is where corners are cut in this communication protocol. The program detects the tone separation, or establishes that one tone switches on/off at a rate of about 1.5 seconds, then it prints the shorthand message.
Joe carries on, stating:
"I have spent much more time on the full-length decoder than the shorthand decoder, which can almost
certainly be improved. I can usually do it myself by eye, if necessary, from the WSJT waterfall display or from Spectran."
Furthermore, in his Powerpoint presentation at the EME conference in 2004 about the source code of the JT65 protocol:
Using "Shorthand tricks for OOO, RO, RRR, 73", and "Shorthand messages are not very robust"
Let me give you an example to make it easier to get my point:
Shorthand messages are decoded only based on the tone spacing, it doesn't matter how these tones switch on/off. This is verified by Joe to be true.
So, when JT65 prints "RO", "RRR" or "73" and the average EME'er thinks that the program has decoded these characters. This is not true, it prints them on the screen when a certain tone spacing is confirmed and no coding is involved.
Actually, instead or "RO" K1JT could have decided to print this message on the screen:
"The computer has now verified that it has detected two tones at XX Hz spacing, and you can proceed by activating TX message #3"
Remember, no message was transferred so there is nothing to decode. Therefore it would be more convenient to help the operator by giving him robust instructions how to proceed.
But, had he done that, people would have said;
The answer is yes. The shorthand message is no "message". Code for RO, RRR or 73 is not transferred over the air. There is no FEC (Forward Errror Control) at all involved, and we are actually seeing the absolute minimum amount of FSK coding, i e tone spacing without need to find out their on/off pattern.
|
K1JT states the following, diminishing the necessity for a report during a QSO to a trivial message (!): "The shorthand messages in JT65, which can be easily decoded by eye or ear, convey just a few "bits" of information. (Is RO being sent? Is RRR being sent? Is 73 being sent? Yes/No on each possibility.) Two bits actually suffice; that's all the information needed to convey the sense of these almost-trivial messages."
|
display and say "Complete QSO!!". As K1JT says, this is not a robust communication protocol, two bits transferred, and no way to check that it is correct is not enough.
However, we understand why he chose to use them. Check this out, info from K1JT:
"Careful measurements on single transmissions with JT65 show that
its messages are decoded reliably yielding 100% copy at the
following approximate signal levels in a 2500 Hz bandwidth:
1. Call1 + Call2 + Grid -22 dB
2. OOO added to above -28
3. Plain text, 13 chars -20
4. Shorthand messages -28"
The table clearly shows that a "genuine" decode of known information needs -22dB S/N,
unknown text needs -20dB (about 1 dB better than the old JT44) and the tone separation shorthand message yields a good -28dB...
In my opinion, this in no way fulfills the requirement of "exchange of unknown information" that the current EME protocol requires. I am hopeful, given what K1JT has told us about the shorthand messages, that future versions will correct this.
Signal reports
First of all, they are averaged over the full receive period. EME signals suffer from QSB, and the CW operator
normally uses peaks to copy the information needed. So does JT65.
In the CW/SSB world we normally say "You were peaking RST 519"!
K1JT has chosen to talk about an average signal level, which by definition is always going to be lower that the peak value that the program uses to copy the message.
Somehow this looks suspiciously like marketing.
Remember the shorthand message detection where the program would need ONE short peak to establish that there are two tones at an expected spacing. If you then average the signal level to S/N reports of -30dB you surely amaze the operator.
And t
he program would not need much more to confirm that two known callsigns are being received as it can make use of known data to reproduce what is missing. Remember, unknown info needs -20dB S/N to be properly decoded.Levels lower than that rely on signal peaks, and known data.
K1JT admits:
"The FEC code used by JT65 is a low-rate code (r = 12/63) and consequently
it, too, benefits greatly from an ability to "copy on the QSB peaks"."
VK7MO Rex writes:
the AFC on WSJT can follow this on a strong signal - those who drift should be aware that they are
giving up the last few dB of performance(signal reports are misleading in measuring performance as
a drifting signal may decode at -30 dB when its real level is -20 dB).
Conclusion: S/N reports in JT65 are chosen to be displayed in a totally different way than what we are used to see. In my opinion this makes it difficult to establish whether one would have been able to copy a CW signal at the same reported level, because for CW we never talk about average strength. Instead we make use of peaks, and clearly, so does JT65. The bandwidth used to deliver S/N reports in no way reflects the bandwidth at which the tones in JT65 are decoded (4.7Hz). Why this method was chosen I have no idea. My suggestion is to also add "Peak" value to the signal reports. This would be very interesting information for both operators as it will give a better indication of what signal levels can be achieved.
"Transmit sensitivity"
K1JT states regarding the call3.txt and the deep search feature:
"As you know, this will provide about 4 dB more sensitivity, both
transmitting and receiving, for DXpeditions that need to use an extra
DXCC prefix or portable suffix."
Conclusion: T
Coding/decoding of characters
The decoding process makes use of known information and K1JT has provided hard coded prefixes in the software. This might be a problem if a new, unknown special prefix is to be decoded. Special formatting is needed for the information to be transferred, else the decoding proces can only manage 13 unknown characters at -20db S/N. So the extra sensitivity can not be used unless the prefix is known.