GD-77 DMR ID enhancements

posted in: DMR, GD-77, Ham radio | 7

The lack of a decent Callsign + name lookup and display system on the GD-77 has always bugged me.

 

When the GD-77 was first released, the only way to be able to display the callsign, was to add stations to the Digital Contacts section of the CPS.

The big problem with this, is the maximum number of Digital Contacts is 1024, and this includes all the TalkGroups. In my case I have about 40 TalkGroups, which only leaves space for around 980 contacts.

Radioddity released an update to the firmware, where they added a separate feature to store around 10,000 callsigns. (I can’t remember when this was introduced, possibly version 3.0

This used a separate program, called ActiveClient, to download the DMR database and then upload it to the radio.

The actual maximum was 10920 callsigns. Its this number because the amount of memory they use for this is 128k bytes, ( 131072 bytes), and each “record” in the database is 12 bytes long. 4 bytes for the DMR ID and 8 bytes (characters) for the callsign.

131072 divided by 12 is 10922.6 but the block of memory has a “header” which contains a identified code if the letters “ID-V001” and also stores the number of records , so that the first “record” starts at an offset of 12 into the DMR ID memory.

So (131072 – 12 ) / 12 = 10921.66 and Radioddity limit this to 10920 in ActiveClient ,though they could have allows 10921.

 

However, this feature has some annoying problems.

1. Unlike Digital Contacts, the number of characters that were allowed per DMR ID was only 8, whereas Digital Contacts have 16.
This means that only the callsign can be displayed, and I like most other people find having the name displayed almost essential.

2. In a signal is received, the firmware checks the DMR ID database  before the Digital Contacts list.
So if the same ID is both the Digital Contacts and the DMR ID data, then the 8 character DMR ID data is displayed, rather than the 16 character (callsign + name) from the Digital Contacts

 

I suspect that Radioddity chose to use 8 characters for the callsign, because this would allow them to advertise that the radio could store “over 10,000” callsigns.

If they had allocated 16 characters for the callsign, to allow space for the name, the number of records would have been 6553, which would have not looked so good in their marketing materials.

 

But for me, I’d rather have 6000 callsigns + names then 10,000 callsigns on their own.

So for the last week I have been looking for the part of the firmware which handles the DMR ID database lookup, and I eventually found it, by tracking down the memory accesses to the part of the memory which we already knew contained this data

I’ve put the C decompilation of this function, created by Ghidra, at the bottom of this post.

 

The function is pretty easy to understand if you know that the value 0x0c is 12 in decimal. This value is both the offset from the start of the memory block and also the lenght in bytes of each record.

 

So calculations like

 

startOffset = iVar4 * 0xc + 0xc;

 

are calculating the position of an individual record within the data block.

 

In theory it looks as if its a simple change to the DMR ID to handle 16 characters per record, simply by changing occurrences of 0x0c to 0x14 (both numbers being in hexidecimal)

The slight confusion is that the offset value is also by coinsidence also 0x0c, and those must remain unchanged.

 

However, just changing the appropriate 0x0c values to 0x14 would not work.

The reason is that the firmware needs buffer space to store 16 characters, and it only has space for 8 characters.

Specifically these lines.

uint local_44;
undefined auStack64 [8];
uint local_38;
undefined auStack52 [8];
uint local_2c;
undefined auStack40 [8];

 

But unfortunately, its not possible to change these [8] to [16], because in Ghidra, and as far as I know, all other reverse engineering programs, the C decompilation is for viewing only, and it does not have a re-compile feature.

 

So the only way to change these values is to change the assembly level instructions; which are much harder to understand.

 

To cut a long story short; its taken me about 3 days to learn enough about the ARM microprocessor assembly language, and to analyse the machine code, to be able to modify the function and change these values to

 

uint uStack92;
undefined auStack88 [16];
uint uStack72;
undefined local_44 [16];
uint uStack52;
undefined auStack48 [16];

 

Note the names have changed, becuase they are partially reflective of the locations in memory (relative to the “stack”).

 

After changing the buffer size, I patched the firmware with this change, and uploaded it to the GD-77, and confirmed that the DMR ID was still working after the change to the buffer sizes.

 

The next step was to modify my CPS to optionally use 16 character DMR ID records (20 bytes in total), and also to modify the firmware to handle both the mathematics of the calculations for 20 byte records and also to read 20 bytes instead of 12 from the SPI Flash memory.

 

Finally, this evening, I’ve just finished both changes, and my new firmware now supports the extended DMR ID record length.

 

But, I still have some more work to do, because I need to change the firmware so it can handle both the old and the new record sizes, because initially after the firmware is updated, the GD-77 could have data in the old format, still in its memory.

 

I hope to be able to achieve this by, changing the header on the DMR ID data.

Radioddity’s programmers included a handy version number of “001” in the header, and currently the existing firmware checks the first 4 letters of this, to confirm whether any DMR data has been loaded at all

i.e the “ID-V” part (actually it also checks for “ID_V” )

 

Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00030000 49 44 2D 56 30 30 31 00 83 0A 00 00 35 42 34 02 ID-V001.ƒ...5B4.

 

But because there is no space between functions in the GD-77 firmware, I can’t simply add some additional machine code instructions.

What I need to do instead, is to overwrite some of the existing check function, and probably use the area of the machine code which checks for the “-V’ or “_V”, and check whether the last digit of the number is 1 or 2.

i.e set the record length at 12, and then if the 7th byte is “2” then add 8 to the record length.

 

 

Once I’ve done this, I’ll encrypt the firmware and send it out some people who have spare radios, which they have told me they will be willing to use as test-beds for the new firmware.

 

Once beta testing is complete, I will of course publish the new version, so as they say… Watch this space…

 

 

———————– Decompilation of the DMR ID lookup function ——————————————–


undefined4 DMR_DB_lookup_000340c6(int iParm1,undefined4 uParm2,byte *pbParm3)
{
int startOffset;
uint uVar1;
int iVar2;
uint uVar3;
int iVar4;
int local_50;
char local_4c;
char local_4b;
char local_4a;
char local_49;
uint local_44;
undefined auStack64 [8];
uint local_38;
undefined auStack52 [8];
uint local_2c;
undefined auStack40 [8];
if (((((pbParm3[4] == 1) && (SPI_Flash_read_area_000263a6(4,0,&local_4c,8), local_4c == 'I')) &&
(local_4b == 'D')) && ((local_4a == '-' || (local_4a == '_')))) &&
((local_49 == 'V' &&
(SPI_Flash_read_area_000263a6(4,8,(undefined *)&local_50,4), local_50 != 0)))) {
uVar1 = (uint)pbParm3[3] |
(uint)pbParm3[1] << 0x10 | (uint)*pbParm3 << 0x18 | (uint)pbParm3[2] << 8; uVar3 = local_50 - 1U & 0xffff; SPI_Flash_read_area_000263a6(4,0xc,(undefined *)&local_44,0xc); if ((int)uVar3 >> 1 == 0) {
if (uVar1 == local_44) {
LAB_0003419c:
memcpy_0001f2c0(iParm1,(int)auStack64,8);
return 1;
}
}
else {
SPI_Flash_read_area_000263a6(4,uVar3 * 0xc + 0xc,(undefined *)&local_2c,0xc);
if ((local_44 <= uVar1) && (uVar1 <= local_2c)) { if (uVar1 == local_2c) { memcpy_0001f2c0(iParm1,(int)auStack40,8); return 1; } iVar2 = 0; iVar4 = (int)uVar3 >> 1;
if (uVar1 != local_44) {
do {
while( true ) {
while( true ) {
startOffset = iVar4 * 0xc + 0xc;
if (iVar4 == iVar2) {
SPI_Flash_read_area_000263a6(4,startOffset,(undefined *)&local_38,0xc);
if (uVar1 != local_38) {
return 0;
}
goto LAB_000341fa;
}
SPI_Flash_read_area_000263a6(4,startOffset,(undefined *)&local_38,0xc);
if (local_38 <= uVar1) break; uVar3 = iVar4; iVar4 = iVar4 + iVar2 >> 1;
}
if (uVar1 <= local_38) break; iVar2 = iVar4; iVar4 = (int)(uVar3 + iVar4) >> 1;
}
} while (uVar1 != local_38);
LAB_000341fa:
memcpy_0001f2c0(iParm1,(int)auStack52,8);
return 1;
}
goto LAB_0003419c;
}
}
}
return 0;
}

7 Responses

  1. Anonymous
    |

    Roger,

    Fantastic job.

    With reference to displaying the callsign and name, may be need to adding the Talker Alias https://www.etsi.org/deliver/etsi_ts/102300_102399/10236102/02.03.01_60/ts_10236102v020301p.pdf similar to MD380. BrandMaister provides Talker Alias data and thus the radio can display this data.

  2. Jørn
    |

    Great work Roger.
    I have 2 spare GD-77 to play with, so if you need beta testers just let me know 🙂

  3. ken
    |

    wow cant wait. cheers

  4. Roger Clark
    |

    I think this enhancement may take some time to evolve.

    I am thinking about ways to squeeze more records into the database by using some form of compression, but there are trade offs no matter which way I do it.

    E.g. If I don’t allow accented charcaters and use automatic capitalisation of the name, I can increase the number of records by about 25%.

    I could try using variable length records, but that would require another 3 bytes per record, and I doubt that would give much of a net gain.

    Ideally I need to free up more of the 1M byte SPI Flash chip to store the DMR ID, and move the display fonts to the microprocessor (MCU) internal ROM and possobiy eve move the calibration data to the MCU ROM.

    There is a lot of scope for improvement but unfortunately, there seem to be very few people with the necessary skill set to help with this

  5. Roger Clark
    |

    Sounds cool, but at the moment I don’t know where any of the DMR data functions are located.

    I want to make some “basic” changes first, before I try for more complex enhancements.

  6. Bevan
    |

    Hello Roger,

    I don’t have a GD77 but still see the benefits of what you are trying to achieve. Just putting this idea out there to see if there is any merit, could the amount of storage space required for channel allocation be reduced if the concept behind the TG channel allocation was thought about differently. Currently it is my understanding that if a user wanted to have say 16 TG’s programmed into their radio for use on say 10 repeaters, they would need to program 160 “Channels” into the radio. If you go away from this method and go to more like a “relationship” database, ie on table of TGs, and one table of Repeater Frequencies. In this way you you would need only 16 records in the TG Table and 10 records in the Repeater Frequency table, even if the records took at the same amount of data that the previous “Channel” record, you would only need 27 records (one record to remember which tg on which frequency the user has currently selected) as apposed to the 160 records, a 80% savings in the data allocation. Granted there will have to be some new logic to select frequency and TG based on user selection. The other advantage to this method could be that radios with GPS functionality could potentially select the nearest repeater or do some method of voting the strongest repeater within range and still be on the same TG that the user had selected.

    Regards
    Bevan
    VK5BD

  7. Roger Clark
    |

    Hi Bevan

    I agree. Its always struck me as a work-around, having to create multiple channels each with the same frequency etc, and only differing in the TalkGroup that is assigned.

    What is needed is a list of TalkGroups that can be assigned to each channel, i.e almost exactly the same as an Rx Group.
    In fact the Rx Group system could probably be repurposed to do this.

    Another option is simply to allow TG entry via the keypad, i.e have a default TG for that channel, but let the user enter an alternative TG if they want.

    The direct TG data entry is one of the best features in MD-380Toolz.

    I’m not sure about the GPS stuff because the GD-77 doesn’t have GPS. And I don’t think simply calculating the nearest repeater is always the best one to use, because of obstacles in the way etc.

    Currently I’m focusing on enhancing the DMR ID stuff, and I’ve got it partially working, using 16 characters per ID. (like digital contacts).
    But when I try to increase the memory allocation, its currently going wrong and not displaying the name at all

    I have started to look at the direct TG data entry at the same time, perhaps using the * (star) key before the number is entered to trigger the radio into TG entry mode, but it looks like its a major hack to the code, and I can’t simply patch some machine code instructions to achieve this, so I’m going to need to start to write code in C and work out how to patch new functions into the existing firmware, by using some space currently used for the Chinese language support.

Leave a Reply