GD-77 Channels export and import

posted in: c#, DMR | 8

I’ve been giving a lot of thought to the best way to handle exporting and importing of channels into the GD-77 over the last few days, and I think I’ve finally come up with a reasonable idea to make it easier to export and import all the information that will allow channels to be exported and then imported with the minimum of hassle

 

Note. The changes I describe below have not been released yet as I’m still testing them…

  • I’ve changed the Channels export to export all data associated with each channel (not just the limited subset which the CPS currently exports). This means that channels that are imported are not missing vital data needed for DMR operation.
  • I’ve modified the Import feature so that it does not clear (delete) all channels before importing.
    My new version, checks each channel as its imported, and if it matches an existing channel name, the existing channel is updated with the revised information from the CSV data.
  • I’ve add (actually re-enabled a hidden feature), that allows groups of channels to be deleted.
  • I’m going to remove the “Clear” button, as this is basically superseded by the “Delete Selected” button, as multiple rows can be deleted by clicking on the first row and shift clicking on the last row.

 

During the import, its currently possible to for a channel to use a Contact name, which is not in the list of digital contacts. This currently causes the “Contact” to be set to “None”. The same applies to Rx Groups and Scan lists.

 

In these cases, currently you would need to manually change these channels to select the correct Contact, Rx Group and / or Scan list.

 

However, I had a brainwave this morning, and I’ve realised that…

 

Ideal 1 (Contact data and Rx Group lists)

If I include the Contact Type and Contact Number, as part of the channel information, that I can create (add) new contacts (TGs or PCs), when a channel is imported.

 

Also if the RX Group referenced by a channel does not exist, I can use the Contact data to either find a Rx Group with that Contact in it, or possibly make a new Rx Group.

 

There is a slight problem with this method, because if I create a new Rx Group with the Contact assigned to the channel that’s being imported.
When the import function encounters the same Rx Group name again, it would presume that it was a pre-existing fully populated Rx Group list and not one that was created by the importer.

 

Probably the best way to overcome this issue is to check that the Rx Group specified for a channel, contains the Contact (TG) used by that Channel.
And if the Contact is not in the Rx Group, then try to add it to the Group.

This is not foolproof, because an Rx Group can only contain 32 Contacts, and a mall-formed CSV (created manually) could have omitted to correctly set the Rx Group.

Hence some error checking will need to be added to the import system to handle this, and perhaps abort the import if that occurs.

 

This will slow down the import process slightly, but I suspect the difference in speed will be barely noticeable on most modern PC’s

 

Idea 2 (Scan lists)

I don’t personally useĀ  Scan Lists, and I’m not sure why the Scan list needs to be specified in the Channel data at all.
Perhaps someone can explain why this is necessary….

However, if the importer encounters a Scan list name that is not an existing Scan list, then a new list could be created.

 

What I don’t know is whether the Scan list should have that channel in it, because at the moment the CPS does not seem to enforce this.

 

Idea 3 Zones

If the channel is in at least one Zone, I can include the Zone name in the data (CSV line) for that channel when its exported.

Then when importing each channel, I can check of a Zone exists with that name and if it does not, the importer can create the Zone and add the channel to it.

Like Rx Groups, if the Zone already exists, I’ll need to check that the Channel is in the Zone, and if not, I’ll need to add it.

 

 

I think if I make the changes described above, it should be possible to do almost a complete import, by just having a spreadsheet containing this augmented channel data.

 

It would not be able to handle instances when the same channel was in multiple zones or multiple scan lists, but I intend working on having separate systems to export and import both Zones and Scan lists (and probably Rx Groups, just for completeness).

And if all else fails, Colin G4EML has a very good Codeplug file < —- > multiple CSV system, which can be used instead.

8 Responses

  1. Wilhelm Onken
    |

    That sounds like a clever idea.

  2. Roger Clark
    |

    Thanks

    I’ll give it a try and post when I have some results.

  3. ken
    |

    sounding great

  4. Roger Clark
    |

    Interestingly, I’ve just written the code to export data in the way I suggested, and it showed up some problems in my existing codeplug.

    I found I had some DMR channels with the Contact set to None, and I’ve also found some different channels which were not in a Zone.

    I initially thought my Export system was not working correctly, but when I double checked, I found my Export worked fine, but my codeplug was broken in places (and some channels would not have worked correctly)

    I’ll now need to work on the Import, which will take a bit longer, as its a lot more complicated than the Export !

  5. James
    |

    Wow. This is really good.

    I would like to test the new version of the CPS if possible as I’m currently generating data for CPS import.

    I already generate working CVS tables for import into the CPS and have found relational references were needed. What I do is simple: I pull the current repeater data from radioid. Then genrate a list of DMR repeaters by CCS7 Id using the band (2m/70cm) and geospatial extents based on proximity to cities and major travel routes. I then generate the CSVs for the contacts, RX Groups (which have to be created by hand), channels, and Zones (which have to be created by hand). The data must be precisely imported/provided to the CPS in that order.

    I generate the callsign lookup table from a private database of lastheard data collected from various MARC C-Bridges and Brandleister TGs. The callsign lookup table takes into account the calls to specific talk-groups (based on the channel data) and the frequency of callers to (theoretically) produce a lower cache miss by eliminating low probability callsigns.

    What would be ideal is to create a metadata standard for radio data. I would be happy to write an open source library of methods that a CPS could use to import and apply data to this standard in the generation of specific code plugs for a radio. It doesn’t make sense that amateurs should be encumbered with the commercial requirements (vertical barrier) to programming these radios.

    Thank you for all your contributions in supporting this radio.

  6. Roger Clark
    |

    I did have the idea about wrapping up meta data etc, but I think for most users, my proposed method of attaching a bit of extra data to each Channel would be sufficient

    I’ve also considered changing the format in which the codeplug is saved to disk e.g. use XML, but it would not make it easy to edit unless there was a separate tool.

    I was going to build the same sort of system that Colin G4EML has written to export spreadsheets for all sections of the codeplug, and I have his source code; but its in Visual Basic and the CPS is written in C#, and converting it proved to be a pain.

    I’ve also considered writing all sections of the codeplug into the save CSV file by adding a row type field to the start of each row.
    I may still do this in the long term, but I want to get the enhanced version of the Channels export and import working first, as I think it would cover 90% of all usage.

  7. Anonymous
    |

    Although I would prefer JSON, I agree with you that it seems CSV files are easier for people.

    Have you seen this code?

    [LINK REMOVED]

    You might be able to reuse or get some ideas here. I use this to automate the programming of the radio.

    Thanks again for all the great work you’ve done.

  8. Roger Clark
    |

    I did consider both JSON and XML as export formats but they are no use to anyone except if they are a programmer.

    The purpose of the Export/ Import feature in the CPS is for people to easily be able to access and manipulate the data, and the common format to do this is CSV.

    I did consider whether XML or JSON could be used, but in addition to it being useless to most users, the file sizes are much bigger than CSV , unless non descriptive property names are used.

    Re: Other tools

    You linked to another tool, but its not a CPS, its a uploaded / downloader

    There are many other tools available for various radios, including the exporter / importer by Colin G4EML as well as loads of other CPS’s for other radios like the MD-380.

    I do not want to confuse this thread by discussing non GD-77 CPS export / import related topics

Leave a Reply