Building a Amateur radio DMR contacts list

Building a Amateur radio DMR contacts list

posted in: Ham radio | 0

One cool feature of Amateur radio DMR, is that the other station’s details can often be shown on the transceiver’s display.

The trick is to have the other station’s details installed as a Contact in the transceiver’s Codeplug database.

 

Update.

With firmware version 3.0.6 (and newer), a new feature was added to separately upload , up to 10920 “DMR ID”s. This feature is not integrated into the main CPS, and uses a separate program called ActiveClient and requires the transceiver to be turned on and connected to USB while several buttons are held down.

In some respects this feature superseded the information in this post, However….
There is one drawback with this feature. Radioddity have only allocated 8 characters for the callsign, rather than the 16 characters available for the “Digital Contact” in the CPS, and there is only enough characters for the callsign and not enough to include the name e.g. a Digital contact can hold “VK3KYY Roger….”, where as the DMR ID system can only hold “VK3KYY R”.
So IMHO, its not as cool, as stations have to give their callsign.

 

Anyway. To continue my original post, assuming you want to make the most of the 1024 Digital contacts which can store callsign and name..

 

This sounds great in theory, however the problem is that the entire DMR-MARC database, has over 83,000 entries, at the time of writing this post, and most transceivers can only store a much smaller number of Contacts.

My Radioddity GD-77 (aka Tytera MD-760) can only store 1024 Contacts using the current firmware (v3.0.6), and 15 of those “Contacts” are taken up with TalkGroups, so I only have space for just over 1000 Contacts

 

Filtering the DMR-MARC database to only include Australian “VK” callsigns doesn’t help because there are around 1300 listed (at the time of writing), and I expect the number will rise as more people buy these cheap handheld transceivers.

The question is which 1000 of the 1300 to include and one possible solution, which I’m currently using is to use the DMR-MARC “Last Heard” page http://ham-digital.org/dmr-lh.php with a filter for all callsigns starting with VK

 

e.g.

 

http://ham-digital.org/dmr-lh.php?callsign=VK*

 

By default page only shows the last 30 entries, however the n=xx parameter allows the maximum number of entries to be changed to much bigger numbers e.g 1000 or more.

Several Caveats.

There is a big problem caused by the paginated form in which the data is available. On several occasions the data has changed between page 1 and page 2 etc, so that a callsign (ID) has been lost. The method described below only works correctly when there is no traffic for the callsigns in question e.g. VK* while all necessary pages are being retrieved.

 

Even though the DMR-MARC database has over 1300 ID’s listed for Australian stations, the “Last heard” page only seems to be currently listing around 708 entries. Its possible that some people have registered ID’s and never used them, or used them prior to 2014, which seems to be the oldest date in the Last Heard system, but I currently think its the best option to get a list of the most active ID’s

 

 

Currently this page is only available as HTML, and also it is paginated with the maximum number of entries per page of 100, so initially,  I manually cut and paste the contents of each page, into Excel, and removed unwanted columns, however as new callsigns are being added to the DMR-MARC database all the time, I would need to do this about once a week, and I have better ways to use my time 😉

 

As a solution to this problem, I decided to write a PHP script to “scrape” the “last heard” page and output the result as a CSV file.

I thought this would be relatively straight forward, as PHP has a DOM object which can parse HTML pages into nicely accessible objects, but I was wrong… The “Last heard” page will not parse because the HTML authoring is invalid in multiple places, and the only way to extract the data was to treat the whole page as a string, and search for specific markers.

This is highly likely to break if the page HTML is modified, but I think even using the DOM object would only be marginally better.

 

I’ve put the code in the scripts folder, and an example usage would be

 

http://rogerclark.net/scripts/dmr-contacts.php?callsign=VK*&limit=1000  (All VK callsigns limited to 1000 entries)

http://rogerclark.net/scripts/dmr-contacts.php?callsign=F*&limit=1000 (all F callsigns, limited to 1000 entries)

 

The format is an index number in the first column, Callsign and name in the second column and ID in the last column. This format is not to suit any specific transceiver, so you need to append any additional columns that your transceiver software needs, and you’ll probably need to append your Talk Groups and re-number the file, to suit.

But its a lot easier than cutting and pasting from the HTML.

 

Update.

As I have a GD-77, I’ve updated the script to include a “radio” parameter e.g.

 

http://rogerclark.net/scripts/dmr-contacts.php?callsign=VK*&limit=1000&radio=gd-77

 

Currently the only “radio” option which does anything is “gd-77” (or “GD-77”), and this format is only compatible with the CPS 2.0.5

(Note. For older CPS versions, the last column has to be renamed from “Call Receive Tone” to “Rx Tone”)

 

This is my current code, which works, but is not exactly elegant 😉 and is available for on Github a https://gist.github.com/rogerclarkmelbourne/1612a2455bf68d45a1fb6f03093e1210

 

 

Update 18th Jan 2018

 

I have been experimenting with using the DMR IR system in conjunction with the Digital Contacts, and unfortunately the firmware in the GD-77 does not do the logical thing.

 

If I load the most active 700 VK callsigns (including names) into the Digital Contacts, and then download the full list of VK DMR ID’s , I’ve found that the name is not being displayed.

Presumably this is because the firmware in the GD-77 is searching the DMR ID dataset first 🙁

Ideally the GD-77 firmware would search the Digital Contacts first and would therefore display the callsign and name.

Currently my workaround is to download the entire database from DMR MARC, filter for VK* then remove ID’s from this list which are in my main Digitial Contacts.

But this is is a laborious process, and needs to be automated.

 

 

Leave a Reply