GD-77 firmware enhancement to Rx Group

posted in: DMR | 36

I’ve just finished another modification to the GD-77

 

The modification allows the DMR channels to work, even when the Rx Group list is set to None, or if the Transmit “Contact” TG is not in the Rx Group.

 

The reasons I’ve made this change are…

 

  1. I need the radio to be able to receive on the Tx TG, for when I modify the firmware to allow direct entry of the TalkGroup.
    Note. I’ve not even started to look at this, because the change to the Rx Group stuff has taken me several days to shoehorn into the existing software
  2. I think the firmware should always have done this..
    It seems crazy that you can setup a channel with a Tx TG and can transmit, but can’t hear incoming signals on the same TG that you transmitted on.

 

For anyone interested in the technical details.

The official firmware has code which compares the incoming signal TG, with each Rx Group list TG in turn, and if there is a match it allows the signal to be received.

The change I made, additionally checks whether the incoming signal is the same as the Tx TG, and it does this check prior to checking the Rx Group.

 

The problem with this change, was trying to optimise the assembly programing, so that I don’t waste the precious space that is available in the ROM.

The ROM is already 99% full, and even if the Chinese fonts are removed, it only gives around 32k for all enhancements. This is not very much at all.

 

I noticed that the internal data structure , which is stored in the RAM of the GD77, has the Tx, TG in the memory location directly before the start of the Rx Group list.

So initially, I thought I’d simply change the code to start checking TG’s from the Tx TG memory location, and increase the number of locations it checked, from 32 to 33.

32 being the max number of TG’s in a Rx Group list.

However I found that the Tx TG, memory location, not only contained the TG, but also some other information, which is either the TG type i.e Private Call, Group call or All Call, or the data may also be the Timeslot (TS1 or TS2)

Regardless of what the additional data is, the problem is that this additional data encoded into the Tx TG memory location, stops me directly comparing the Tx TG with the incoming signal TG, as they won’t ever be the same number.

So I had to “mask off” the portion of the 32 bit memory location which contained the additional information.

I found there wasn’t space within the existing Rx Group checking code, so I had to make a sub routine (aka function), which stripped off the additional information before giving the sanitised TG back to the existing checking part of the code.

This worked OK, but I didn’t feel that it was an optimal solution, because the masking function was being run for the Rx Group list TG’s which didnt need to be masked, and would slow things down a tiny bit.

 

So I decided to program it correctly, and to check the Tx TG prior to the code checking the Rx Group list TG’s

 

Looking at the assembler code, I found the IAR compiler that Raioddity use, had 4 duplicate assembler instructions, which I could remove, from the core of the Rx Group checking system, and using this space, I was able to move all the prior instructions down, into the free space, which left me with 12 bytes to play with, to check the Tx TG prior to the Rx Group checking code.

It was actually quite time consuming to “move” the code downwards, as I had to retype about 50 machine code instructions into different locations, and also fix up the branch addresses to match the new code layout.

This too several hours.

Then with the 12 instructions prior to the existing code, I was able to “call” a new function, which loaded the Tx TG, and masked off the Call Type, and Timeslot data, so before returning the sanitised TG to the existing code.

And with the remaining 8 bytes, I could compare this TG with the incoming TG and take the appropriate action, i.e either accept the signal immediately, or get the program flow to continue onto the existing Rx Group checking code.

 

It took me about 3 or 4 attempts to get this new code to work, as I had a few minor bugs which prevented it from working at all.

But it finally worked OK.

 

I’ve sent this version via email to a few people to test, and I’m hoping to get some feedback over the next week.

 

And after a short break, I’ll start to look at the Direct TG entry system

 

Update Saturday 6th April

 

Rick, W1RHS  has alerted me to a possible side effect of this change.

On his hotpot, the audio amplifier does not seem to be getting muted during the tail of each transmission, when his hotspot continues to transmit, but there is no audio in the DMR signal.

 

I have tested this myself and I can’t replicate the problem, but it may depend on the hotspot settings.

 

My guess is that, the firmware doesn’t think it needs to mute the audio, because it doesn’t think the amplifier was unmuted in the first place.

 

So I’ll need to hunt down the part of the firmware which handles the end of an incoming transmission and probably always mute when the incoming audio ends.

 

Update – Saturday 7th April

 

I’ve tracked down the noise problem, and it wasn’t what I thought it was…

The problem was the vocoder, trying to process empty data frames at the end of an Over, which caused the “aaaaa” sound that sometimes happens in the middle of an over if there is some sort of interruption in the signal.

The problem seemed to be that the Rx Group list, was still being used at the end of an incoming Over, so I took another look at the code and to my surprise I found another 4 places where the Rx Groups were checked.

But rather than centralizing the RxGroup checking functionality, the programmers had simply cut and paste the same 5 lines of C code (about 25 lines of assembler), in 5 different places.

This is really sloppy programing, even if you have the C source code, which I don’t, because it means that if there is a bug in those 5 lines of C code, you have to remember to fix it in 5 places not 1.

Also its wasteful, as it takes around 200 bytes extra of ROM space. If they have been this wasteful elsewhere in the firmware, I’m not surprised that the GD-77 uses around 99% of available ROM space for its firmware, and this could probably be reduced by 10% at least.

 

Anyway. I’ve now rewritten the RxGroup checking into its own sub-routine (also know as a Function), and patched the places that had the duplicate lines of programming, to all use this one central function.

I’ve sent the file to a few people and they have confirmed that the bug now seems to have been fixed.

 

If I get any time during the week, I’ll start to look at the Direct TalkGroup entry system.

36 Responses

  1. W1AEX - Rob
    |

    Nicely done Roger! The RX Group debacle really had me stumped before a friend informed me that the GD-77 was “different” from other DMR rigs that knew enough to use the TX talkgroup when the RX Group field was set to “none”. You are doing an amazing job of tracking down and correcting the biggest GD-77 shortcomings that so many of us “users” have struggled with. Thank you for all the long hours you have put into resolving the GD-77 puzzles! 73, Rob W1AEX

  2. F1CXG
    |

    Great job Roger 😉
    Congratulations for the perseverance to improve the GD77

    F1CXG Thierry

  3. Jørn
    |

    Wow great work Roger!
    If you want me to test, let me know.

  4. Roger Clark
    |

    Hi Jørn

    I’ll email you a link to download the firmware, because I already uploaded it to an Issue in github.

    BTW. Rick W1RHS has found a problem with it on his hotspot, with some audio noise after the end of the other station’s transmission.

    The problem seems only to be occurring since this change, so I think somehow the change sometimes affects the audio muteing

    I’ll update my blog post with this information

  5. Roger Clark
    |

    No problem

    BTW. If you want to test this firmware please let me know, and I will send you a link to download the firmware.

  6. Cesar Silva
    |

    Nice work Roger! can you send me the download link for testing?

  7. ESTEBAN PRADO
    |

    please a curiosity will continue to be worth the same codeplug that the computer has installed. Or everything is in beta and we should wait. Thank you

  8. Brian K1LI
    |

    Being new to DMR, I can’t figure out the purpose of the Rx Group in the first place.

    On the advice of a DMR mentor, I started with a 1:1:1 correspondence between my list of Digital Contacts, Rx Groups and Channels. This allowed me to start making contacts. I frequently see the GD-77’s green LED illuminate but don’t hear any audio. I can click through the Channels until I find the one associated with the appropriate Digital Contact.

    I thought I could fix this problem by adding Digital Contacts to the Rx Group associated with the Channel to which I typically set the radio, but there was no change. I know that I could Scan through the channels, but this doesn’t help me understand the purpose of Rx Groups.

    I know I missed something when they were handing out the DMR brains… what is it?

  9. Roger Clark
    |

    1:1 for TGs ( Digital contacts ) to RxGroups is used by a lot of people, and my latest fix does this automatically.

    Currently there still needs to be one a Channel per Tg, and when you use multiple repeaters this needs to be duplicated for all the repeater frequencies

    I will be implementing a way to directly enter the TG number via the keypad, which will then remove the need for this duplication.

    Another possible way to simplify, the Channels structure would be for me to change the Rx Group to become a TxRXGroup, where you just assign a group to a single channel, and the radio would use the TGs in that new group type to allow the selection of the TG, in the same way you currently use the up and down arrows to change channel.
    But this would take far more work than the direct TG entry.

    However I have a audio gate problem with my latest firmware that I also need to fix.

  10. Roger Clark
    |

    Do you mean Codeplug or firmware?

    The Codeplug contains the Channels and TGs etc
    The firmware contains the functionality of how the radio operates and what feature it has

  11. Roger Clark
    |

    I posted the link here (yesterday) but I’ve since updated the firmware to fix a problem. See my next comment

    Please be aware this firmware is experimental and you use it at your own risk, but it has been tested by around 20 people with no problems.

    It seems to be stable, but there is a problem with the audio gate at the end of an incoming over.
    The work-around for the audio gate issue, is currently to make a new TG called something like Empty, and do not put any TG’s in it.

    Then assign that Rx Group to your channels.

    If you select None for the Rx Group you may get some noise at the end of overs. This depends on your radio and how high the battery is charged. If the battery is below 8V there is normally no noise even with Rx Group of None selected

  12. Chris
    |

    Please send send CFW when you have time also will email you a copy of backup Thanks N3WER Chris

  13. Chris
    |

    Downloaded Thanks BTW no readme you install Firmware just as OEM?

  14. Roger Clark
    |

    Just install using the normal Radioddity firmware uploader.
    i.e the one that can be downloaded with version 3.1.8 or 3.2.1 etc.
    Its the same installer for all Radioddity software

  15. Roger Clark
    |

    Cesar

    I’ve now fixed a bug with my previous version, this is the updated SGL file

    Please be aware this firmware is experimental and you use it at your own risk

    https://github.com/rogerclarkmelbourne/Radioddity_GD-77/files/3051448/gd-77_3.1.8_20190407_RxGroupFix.zip

  16. W1AEX - Rob
    |

    Roger,

    The “gd-77_3.1.8_20190407_RxGroupFix” release is working flawlessly for me since I installed it yesterday morning. I’ve run it for over 18 hours and have noticed no issues at all. With the RX Group List set to “None” it receives the selected TX talkgroup perfectly.

    Additionally, the enhanced DMR-ID feature has functioned beyond my expectations. I ended up with a setting of 13 characters due to the fact that with my GD-77 the 14th character was partially blocked by the phone icon. I ended up with a maximum of 15419 possible contacts and by reducing the inactivity filter to 75 days I was able to install regions 11 + 31 +10 and then region 302 set with a 30 day inactivity filter. This gave me the complete US + Canada DMR-ID listing (15394 contacts with the mix of 75 and 30 day filters applied) which works very well for my uses.

    Thanks again and 73,

    Rob W1AEX

  17. Roger Clark
    |

    Thanks Rob

    I am glad it’s now working OK for you.

    Re DMR ID

    I think there will be scope to make better use of the 1M byte memory, but it will take some time as I will need to effective defragment the memory, which will involve changing a lot of the code.

    BTW.

    It sounds like I need a way to allow multiple regions to be entered , e.g. separated by a comma, so that you would not need to keep exerting each region individually and then downloading.
    I can save the regions numbers list, so it would be restored next time you go to update the IDs

    I will have a think about the best way to do this ..

  18. W1AEX - Rob
    |

    “I can save the regions numbers list, so it would be restored next time you go to update the IDs”

    Roger – This would be a wonderful convenience, but it sure sounds like a lot of work for you to sort it out!

  19. Roger Clark
    |

    I will put it at the bottom of my To Do list 😉

  20. W1AEX - Rob
    |

    :O)

  21. Jon
    |

    Excellent Work Roger! The two other big bugs perhaps more important than the RX Groups that need to be squashed are making the Admit Criteria work properly and The Talk Permit Tone to work properly. With these things fixed the GD-77 will be a much more usable radio.

  22. Roger Clark
    |

    Since I don’t have the source code, I can only start to fix parts of the firmware binary, which I can track down as doing specific functions.
    The Rx Group (none) thing, is one case in point, because the Rx Group list data can clearly be seen when I dump the RAM when the radio is running.

    Things like the Admit criteria are a different level of problem entirely.

    At the moment, no one has isolated any of the low level DMR data processing functions which would be involved in this.

    The scale of the task is vast.

    There are at least 5000 “functions” in the code, and after 1 month of solid effort by 2 people, there are still 2000 functions which we have no idea about whatsoever.

    With the remaining 3000 functions, we generally only know roughly what area of the firmware they are in.
    e.g. There are around 250 audio decode/encode functions, around 300 functions associated with the menu system
    etc etc

    So unless more people are willing to devote their time and their hardware (since its essential to take the radio apart and solder a hardware debugger to the PCB), things are not going to happen very quickly.

    Currently I’m the only one with a hardware debugger attached to their radio, as one one else wants to damage theirs.

  23. Bob
    |

    Hi Roger…..downloaded your firmware and replaced version 3.2.1 in one of my GD-77s and running like a champ. Usually runs through a hot spot in my office but have plenty of local repeaters available as well. I compare it to my other GD-77, MD-380 and MD-390 and it hasn’t skipped a beat. Nice to have a name along with the call from DMR ID. Thanks.

  24. David
    |

    Will this project also include the GD-77S in the future. I know it is a lot to ask, but the GD-77S seems to be the goto for the sight impaired and any improvement would make the unit operation easier would be a big help. Simple things like being able to interrupt the Channel Info dialog and restoring the decimal to the left of the decimal point which was apparently considered of no use. Always great to pull a fresh battery off the charger and have the GD-77S report it as “Point 4” or drop a significant digit off of the frequency read back. The lack of a display or menu options takes a lot out of the firmware, and there should be room to improve the vocabulary other features. Again, a lot to ask – but planting a seed for consideration of the sight impaired.

  25. Jürgen
    |

    Hi Roger, I need your help on a (https://www.stm32duino.com) topic, but I can´t register there and I have no idea how to contact you via email. Please get in touch with me. Much appreciated. Thanks.

  26. Roger Clark
    |

    I don’t have a GD77S and I would not personally have time to work on anything related to the GD77S.

    Even making changes to the GD77 is already taking a lot more time than I had anticspaied.

    If you know of a developer who is interested in working on the GD77S for you, I can share my Ghidra file for the GD77 which now has some annotations for around 2000 of the 5000 “functions” in the firmware binary file.
    But I suspect the anotations would not be much use for the GD77S as the mainly relate to things associated with the keypad and the display

  27. Roger Clark
    |

    Registration has been disabled at the moment, because I no longer have the time to manage the site.

    I forget the address but a few people have setup clone sites.

    You could try posting to one of them.

  28. EvO
    |

    Hi Roger.
    I think you already know this, anyway about your gd-77_3.1.8_20190407_RxGroupFix POC I noted that while using it when for any reason I do reset of the GD-77 then RF alignment data into the transceiver are quite different from what is listed here http://members.optuszoo.com.au/jason.reilly1/GD-77tune.htm (thanks to Jason Reilly) which exactly corresponds to the values checked with the GD77_FlashManager before proceeding with the reset.
    So by switching from official 3.1.8. firmware released by Radioddity to the improved firmware gd-77_3.1.8_20190407_RxGroupFix POC released by you the RF alignment data survives, but then doing reset of the GD-77 which is running gd-77_3.1.8_20190407_RxGroupFix POC the resulting RF alignment data are incongruous.
    The quickest way to restore RF alignment data is to downgrade to the old firmware GD-77_V2.6.6 by Radioddity, do a reset, update the firmware with your gd-77_3.1.8_20190407_RxGroupFix POC version, load codeplug and DMR-ID and enjoy the transceiver.
    Being able to read the name of the correspondent as well as the name excites me a lot, very useful and funny.
    Then, no longer having to be forced to associate each contact with an RX group make the writing of codeplugs a piece of cake!
    Thanks a lot Roger and please keep it up!

  29. Roger Clark
    |

    The Factory Reset has not worked correctly since version 3.

    It does not reset the codeplug data and it does not restore backup calibration into the Master calibration.

    Since the backup calibration is actually overwritten with possibly random data , if you do a factory reset, I decided to use the memory used by the backup data as it’s right in the middle of a 64k data block which is not used for anything else.

    Radioddity should have provided a way to backup and restore and also to configure the calibration, but they didn’t.

    I probably need to write a program to backup the calibration on its own, but Flash Manager can do this.

    Downgrading to 2.x and doing a factory reset could cause problems, because the official firmware makes changes to the flash memory locations which may be used by firmware 2.x for other purposes.

    In general I would never downgrade to 2.x as it’s far too risky and you could potentially brick you radio, either going back to 2.x or forward again to 3.x

    Jason found that if some memory locations in the flash are incorrect the radio will not bootup correctly and you can’t even enter the mode to use Flash Manager to restore the flash memory.

    He had to remove the flash chip and restore using a hardware programmer.

    I probably need to fix the firmware , so that it can bootup even if the flash memory is completely corrupted.

    However there are loads of other bugs that need to be fixed and I only have a limited amount of time to fix them, and no source code for the firmware

  30. Ted Forgach
    |

    Please excuse me, if you already know about this comment: The time stamp on your “Responses” section of this page shows
    “06/04/2019” and the consecutive responses are also misdated. The time stamp error is a little distracting. I use three browsers: Firefox, Google and Microsoft Edge with Windows 10 and it shows up in each.

  31. Ted
    |

    Sorry, you can disregard my post — your date format must be day/month/year. Silly me! I naturally read it wrong and it looked like June 4, 2019. I’m laughing at my self.

  32. ESTEBAN PRADO
    |

    good morning can summarize the firmware improvements 3.2.1 and if you can put the link for download. I have been offline for some time and I have not read any post. Thanks for your time

  33. Roger Clark
    |

    Firmware 3.2.1 is an official release from Radioddity.

    Its to make the GD-77 compliant with the latest FCC regulations and prevents out of band operation even if the CPS band limits are changed.

    You need to read the Update Diary document provided by Radioddity for full details of this firmware.

    Note. This has nothing to do with any of the firmware changes I’m making.

    None of my firmware changes are available for general download, and are not supported by Radioddity

  34. EvO
    |

    Hi Roger.
    Thanks alot for your kind reply.
    You are totally right, switching to old firmwares for then rewrite with the new ones don’t help, it doesn’t works anymore with the unlocked firmware you provided, Flash Manager did it though.
    In fact I thought I could recover the master calibration using the trick of uploading an old firmware first and then update the transceiver with the current one, but it didn’t work, the master calibration was always and constantly wrong.
    So I saved the day using your Flash Manager with an old backup that I got in the past and I have to write that actually Flash Manager is the quickest and more effective way to restore master calibration, no way!
    About FlashManager, I see there is a checkbox named “Read internal Flash” that can be set or not, but I can’t figure out what is its purpose, what does it do exactly?
    I’d like to undertstand what it is and be sure of doing the backup of the whole flash chip in the right and best way.
    I agree that make changes in the firmware so that it can bootup even if the flash memory is completely corrupted would be very useful.
    Thank you very, very much Roger, please keep it up!

  35. Roger Clark
    |

    The extra checkbox is a feature added by Kai DG4KLU but only works with some special firmware he developed.
    The only purpose of his special firmware is to allow the Bootloader and also the CPU key lock values to be read, because although his firmware allows all of the internal ROM to be read using Flash Manager, his firmware consumes most of the internal ROM, and hence there is no point reading out something which you have just installed.

    I think I will need to remove that checkbox, as its only usable with his special firmware, which only needs to be used when attempting to recover the bootloader in new types of radios.

  36. EvO
    |

    Hi Roger.
    Thank you for your detailed reply.
    I have taken good note of it for future use
    Roger please keep it up!

Leave a Reply