Merge pull request #49 from semuconsulting/RC-1.0.16

RELEASE CANDIDATE 1.0.16

.github/workflows/checkpr.yml

@@ -11,9 +11,9 @@ jobs:
         python-version: [3.8, 3.9, "3.10", "3.11", "3.12"]
 
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
       - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
       - name: Install deploy dependencies

.github/workflows/main.yml

@@ -13,9 +13,9 @@ jobs:
         python-version: [3.8, 3.9, "3.10", "3.11", "3.12"]
 
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
       - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
       - name: Install deploy dependencies
@@ -37,7 +37,7 @@ jobs:
         run: |
           pytest
       - name: "Upload coverage to Codecov"
-        uses: codecov/codecov-action@v3
+        uses: codecov/codecov-action@v4
         with:
           token: ${{ secrets.CODECOV_TOKEN }} # supposedly not required for public repos
           fail_ci_if_error: true

.vscode/settings.json

@@ -4,5 +4,5 @@
     "editor.formatOnSave": true,
     "modulename": "${workspaceFolderBasename}",
     "distname": "${workspaceFolderBasename}",
-    "moduleversion": "1.0.15"
+    "moduleversion": "1.0.16"
 }

README.md

@@ -21,7 +21,7 @@ The `pyrtcm` homepage is located at [https://github.com/semuconsulting/pyrtcm](h
 This is an independent project and we have no affiliation whatsoever with the Radio Technical Commission for Maritime Services.
 
 **FYI** There are companion libraries which handle standard NMEA 0183 © and UBX © (u-blox) GNSS/GPS messages:
-- [pyubx2](http://github.com/semuconsulting/pyubx2) (**FYI** installing `pyubx2` via pip also installs `pynmeagps` and `pyrtcm`)
+- [pyubx2](http://github.com/semuconsulting/pyubx2)
 - [pynmeagps](http://github.com/semuconsulting/pynmeagps)
 
 ## <a name="currentstatus">Current Status</a>
@@ -35,7 +35,7 @@ This is an independent project and we have no affiliation whatsoever with the Ra
 ![Contributors](https://img.shields.io/github/contributors/semuconsulting/pyrtcm.svg)
 ![Open Issues](https://img.shields.io/github/issues-raw/semuconsulting/pyrtcm)
 
-Parses RTCM3 messages into their constituent data fields. Refer to the `RTCM_MSGIDS` dictionary in [`rtcmtypes_core.py`](https://github.com/semuconsulting/pyrtcm/blob/main/src/pyrtcm/rtcmtypes_core.py) for a list of message types currently implemented. Additional message types can be readily added - see [Extensibility](#extensibility)).
+Parses RTCM3 messages into their constituent data fields - `DF002`, `DF003`, etc. Refer to the `RTCM_MSGIDS` dictionary in [`rtcmtypes_core.py`](https://github.com/semuconsulting/pyrtcm/blob/main/src/pyrtcm/rtcmtypes_core.py) for a list of message types currently implemented. Additional message types can be readily added - see [Extensibility](#extensibility).
 
 Sphinx API Documentation in HTML format is available at [https://www.semuconsulting.com/pyrtcm](https://www.semuconsulting.com/pyrtcm).
 
@@ -140,17 +140,21 @@ Example:
 <RTCM(1005, DF002=1005, DF003=0, DF021=0, DF022=1, DF023=1, DF024=1, DF141=0, DF025=4444030.8028, DF142=1, DF001_1=0, DF026=3085671.2349, DF364=0, DF027=3366658.256)>
 ```
 
-The `RTCMMessage` object exposes different public attributes depending on its message type or 'identity'. Attributes are defined as data fields ("DF002", "DF003", etc.) e.g. the `1005` message contains the following data fields:
+The `RTCMMessage` object exposes different public attributes depending on its message type or 'identity'. Attributes are defined as data fields (`DF002`, `DF003`, etc.) e.g. the `1087` multiple signal message (MSM) contains the following data fields:
 
 ```python
 >>> print(msg)
-<RTCM(1005, DF002=1005, DF003=0, DF021=0, DF022=1, DF023=1, DF024=1, DF141=0, DF025=4444030.8028, DF142=1, DF001_1=0, DF026=3085671.2349, DF364=0, DF027=3366658.256)>
+<RTCM(1087, DF002=1087, DF003=0, DF416=2, DF034=42119001, DF393=1, DF409=0, DF001_7=0, DF411=0, DF412=0, DF417=0, DF418=0, DF394=4039168114821169152, NSat=7, DF395=1090519040, NSig=2, DF396=16382, NCell=13, DF397_01(003)=69, DF397_02(004)=64, DF397_03(005)=73, DF397_04(013)=76, DF397_05(014)=66, DF397_06(015)=70, DF397_07(023)=78, DF419_01(003)=12, DF419_02(004)=13, DF419_03(005)=8, DF419_04(013)=5, DF419_05(014)=0, DF419_06(015)=7, DF419_07(023)=10, DF398_01(003)=0.6337890625, DF398_02(004)=0.3427734375, DF398_03(005)=0.25390625, DF398_04(013)=0.310546875, DF398_05(014)=0.5126953125, DF398_06(015)=0.8271484375, DF398_07(023)=0.8837890625, DF399_01(003)=-665, DF399_02(004)=29, DF399_03(005)=672, DF399_04(013)=-573, DF399_05(014)=-211, DF399_06(015)=312, DF399_07(023)=317, DF405_01(003,1C)=0.00024936161935329437, ... , DF404_12(015,2C)=0.3947, DF404_13(023,1C)=0.6146)>
 >>> msg.identity
-'1005'
->>> msg.DF024
-1
+'1087'
+>>> msg.DF034
+42119001
+>>> msg.DF419_03
+8
 ```
 
+Attributes within repeating groups are parsed with a two-digit suffix (`DF419_01`, `DF419_02`, etc.). Attributes within MSM NSAT and NCELL repeating groups can optionally be labelled with their corresponding satellite PRN and signal ID when the `__str__()` (`print()`) method is invoked, by setting the keyword argument `labelmsm` to True - e.g. `DF404_13(023,1C)` signifies that the 13th item in the DF404 ("fine Phase Range Rate") group refers to satellite PRN 023, signal ID 1C.
+
 Helper methods are available to interpret the individual datafields:
 
 ```python
@@ -168,8 +172,6 @@ Helper methods are available to interpret the individual datafields:
 
 The `payload` attribute always contains the raw payload as bytes.
 
-Attributes within repeating groups are parsed with a two-digit suffix ("DF030_01", "DF030_02", etc.). 
-
 **Tip:** To iterate through a repeating group of attributes (*e.g., DF406 (GNSS signal fine PhaseRange)*) as a list, the following construct can be used:
 
 ```
@@ -179,8 +181,6 @@ for i in range(msg.NCell):
     df406group.append(df406)
 ```
 
-Attributes within MSM NSAT and NCELL repeating groups can optionally be labelled with their corresponding satellite PRN and signal ID when the `__str__()` method is invoked, by setting the keyword argument `labelmsm` to True (e.g. `DF405_10(014,2C)` signifies that the 10th item in the DF405 group refers to satellite PRN 014, signal ID 2C).
-
 ---
 ## <a name="generating">Generating</a>
 
@@ -265,7 +265,7 @@ gnssdump -h
 ---
 ## <a name="gui">Graphical Client</a>
 
-A python/tkinter graphical GPS client which supports NMEA, UBX and RTCM3 protocols is available at: 
+A python/tkinter graphical GPS client which supports NMEA, UBX, RTCM3, NTRIP and SPARTN protocols is available at: 
 
 [https://github.com/semuconsulting/PyGPSClient](https://github.com/semuconsulting/PyGPSClient)
 

RELEASE_NOTES.md

@@ -1,5 +1,11 @@
 # pyrtcm Release Notes
 
+### RELEASE 1.0.16
+
+CHANGES:
+
+1. PRN SIG mapping streamlined - `id2prnsigmap()` helper method replaced by dictionary `PRNSIGMAP`.
+
 ### RELEASE 1.0.15
 
 CHANGES:

examples/benchmark.py

@@ -11,11 +11,13 @@
 """
 # pylint: disable=line-too-long
 
+from platform import python_version
+from platform import version as osver
 from sys import argv
-from datetime import datetime
-from platform import version as osver, python_version
-from pyrtcm.rtcmreader import RTCMReader
+from time import process_time_ns
+
 from pyrtcm._version import __version__ as rtcmver
+from pyrtcm.rtcmreader import RTCMReader
 
 RTCMMESSAGES = [
     b"\xd3\x00\x13>\xd0\x00\x03\x8aX\xd9I<\x87/4\x10\x9d\x07\xd6\xafH Z\xd7\xf7",
@@ -27,8 +29,7 @@
     b"\xd3\x00\xc3C\xf0\x00J\n\xbdf\x00\x00\x1c\x07\x01\x00\x00\x00\x00\x00 \x80\x00\x00\x7f\xfc\x8a\x80\x92\x98\x84\x8c\x9d\x9b\n\x0fTJ\xbe\x82'\xd0n\x9f\xc4\xfa\xce\x00\xe8T\x1e\xe1\xfeZ\t\xc0'\xa4\x15\xe6A\xd7_;\xc1\xf2\x85`.\xbe\x05\xa3'\xb6\xa6}\xb2y\xa4\xf5\x9dl\x84\x8a\x98KE\xfc!\xa6\x10W\xc8\x10oM\xfc\xd4\xe9\xfc\xa4<\x00\xbb\x0e\x01m\xcc\x1e\xd1\xb6\x1f\xc6\x0f\xe6\x98\xf1\xe7_4\x126\x18\x12\xe1\x05\xf0x\x14\xaa\xaa\xaa\xa2\xa8\xaa\xaa\xaa\xa2\xaa\xaa\xaa\xaa\xaa\xaa\xaa\x00\x02\xf0\xa0/\n\x82\xf0\x9c$\x08C\x00\xac0\n\x02\x90\xbf\xff\x80M\n\xda\x13S\x94\xa7#\xfb!\xf6\x11\xef%\xdd\xf8z\xa0\xf6\xb3\x00@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00v'\xaf",
     b"\xd3\x00\x91D\x90\x000\xab\x88\xa6\x00\x00\x01\x80\x04\x12\x00\x00\x00\x00 \x01\x00\x00\x7f\xe9\xea\x8b)\xca`\x00\x00P +Z\xf8\x85~u\xef\xe04\xe0\x1f\xfd\x01\xf4\x19\x7f\x89\x81\xa5N:\xa52~\x15h6e\xdc\x18\xdd\xefY\xfb*\x9f\xf3?\xfd\x16Q\xfe$K\xe8\xe5;\xea\x9c\\\x1f\x97D \xd2\xc9\xf6\xfb\xf5\xf7\xb8\x19\xfe\xd1a\xff\xc8\xc2\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xa0\x05\xc1\x88R\x15\x85aXZ\x18\x85axiR\xd2s\x83\xd7\x07\xc9\xc4\xb3\x80\xc5g\x8a\xd4\xe1\xd2\xc3\x96\x00\x08y",
     b'\xd3\x01\rFp\x000\xaa\xad\xe4\x00\x00\x01`\t\x08\x84\x90\x00\x00 \x02\x00\x00/UT\x0c#\xf2Z\x8a\xa2rT\x12\xb0\x00\x00\x00\x00\x00\xf0\xf6\xa7\xb7;I$G\xaaT\xa1Y~\xfd\xfe7\xf5\xe0\x10|\xe4\r\xa7\xbe\xbf\xdf\xfe\x94\x02~h\x96\x0e\xe5\x89\xa7E\x19\xf4\xf7Q\x0e|\xe29\x81Q\x91s\xc6\xf9\x95\xf8C\xae\xcb\xf6\xf9\xa3\xbd\x83\xb5\xfb\x06\x9b"\x86~\xb7}C\xca\x7f4\xa1\x06\x0e\xb2\x84Y6\xfb\xe2\x95~\x0e6{*\xdc\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\x00-\nB\xa0\xb40\x0b\x82\xa0\xbc0\x0b\x02\xb0\xd3\xad\xa0c\xd4\xc7\xac\xc2\xed\x18\xdc\x03bo,\xd1S\x96\xcc\xbfP\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xb2X\xbc',
-    b"\xd3\x00\x04L\xe0\x00\x80\xed\xed\xd6",
-    b"\xd3\x00\x08>\xf4\xd2\x03ABC\xeapo\xc7",
+    b"\xd3\x00\x9dE\xd0\x00[\xfc\x95\x82\x00 1\x00\x00\x00\x00\x00\x00\x00 \x00@\x80\xff\xfcc\x84(\x00\x16\x04\x06\xfa\x0f\x0f\xfc\x80\x00\x8e\xd5q\x13\xcd\x10\x14\xe0\xf1t}z'\xf7D\xfe\x85\x17\xdd\x84y\x92\xdf\xb4\xe5z`\xf7\x9b\xa0\xfcf(\x084\xe1\xff\xefn\x83\xa3\x13\x86dJ}3\x15\xfdD\x03|\xea\xe0\x84\xa7\xe5\xfc\xcaS\x03^\xc9\x02\xe8\xa3@\x8euX\xb6\xd1\x14M\x13D\xc7\x16\x05\x81`\x00\x03\xcb\x1fE\x11F\x15\x94iY\x9c\xcb]\xdb\xb4/\xbd\x88\x00\x10\x00 \x00i\r\x80\x01\x00\x02\x00\x06\xfd\xe8\x00\x10\x00 \x00\x00\x92\x8e\xca",
 ]
 
 
@@ -72,19 +73,19 @@ def benchmark(**kwargs) -> float:
         f"\nTxn per cycle: {txnc:,}",
     )
 
-    start = datetime.now()
+    start = process_time_ns()
     print(f"\nBenchmark test started at {start}")
     for i in range(cyc):
         progbar(i, cyc)
         for msg in RTCMMESSAGES:
             _ = RTCMReader.parse(msg)
-    end = datetime.now()
+    end = process_time_ns()
     print(f"Benchmark test ended at {end}.")
-    duration = (end - start).total_seconds()
-    rate = round(txnt / duration, 2)
+    duration = end - start
+    rate = round(txnt * 1e9 / duration, 2)
 
     print(
-        f"\n{txnt:,} messages processed in {duration:,.3f} seconds = {rate:,.2f} txns/second.\n"
+        f"\n{txnt:,} messages processed in {duration/1e9:,.3f} seconds = {rate:,.2f} txns/second.\n"
     )
 
     return rate
@@ -101,5 +102,4 @@ def main():
 
 
 if __name__ == "__main__":
-
     main()

pyproject.toml

@@ -7,7 +7,7 @@ name = "pyrtcm"
 authors = [{ name = "semuadmin", email = "[email protected]" }]
 maintainers = [{ name = "semuadmin", email = "[email protected]" }]
 description = "RTCM3 protocol parser"
-version = "1.0.15"
+version = "1.0.16"
 license = { file = "LICENSE" }
 readme = "README.md"
 requires-python = ">=3.8"

src/pyrtcm/_version.py

@@ -8,4 +8,4 @@
 :license: BSD 3-Clause
 """
 
-__version__ = "1.0.15"
+__version__ = "1.0.16"

src/pyrtcm/rtcmhelpers.py

@@ -8,28 +8,14 @@
 :copyright: SEMU Consulting © 2022
 :license: BSD 3-Clause
 """
+
 # pylint: disable=invalid-name
 
 from datetime import datetime, timedelta
 
 from pyrtcm.exceptions import RTCMTypeError
-from pyrtcm.rtcmtables import (
-    BEIDOU_PRN_MAP,
-    BEIDOU_SIG_MAP,
-    GALILEO_PRN_MAP,
-    GALILEO_SIG_MAP,
-    GLONASS_PRN_MAP,
-    GLONASS_SIG_MAP,
-    GPS_PRN_MAP,
-    GPS_SIG_MAP,
-    IRNSS_PRN_MAP,
-    IRNSS_SIG_MAP,
-    QZSS_PRN_MAP,
-    QZSS_SIG_MAP,
-    SBAS_PRN_MAP,
-    SBAS_SIG_MAP,
-)
-from pyrtcm.rtcmtypes_core import RTCM_DATA_FIELDS, RTCM_MSGIDS
+from pyrtcm.rtcmtables import PRNSIGMAP
+from pyrtcm.rtcmtypes_core import RTCM_DATA_FIELDS
 
 
 def att2idx(att: str) -> int:
@@ -101,21 +87,6 @@ def bits2val(att: str, scale: float, bitfield: int) -> object:
     return val
 
 
-def num_setbits(val: int) -> int:
-    """
-    Get number of set bits in integer.
-
-    :param int val: integer value
-    :return: number of bits set
-    :rtype: int
-    """
-
-    i = 0
-    for x in bin(val)[2:]:
-        i += int(x)
-    return i
-
-
 def calc_crc24q(message: bytes) -> int:
     """
     Perform CRC24Q cyclic redundancy check.
@@ -303,7 +274,7 @@ def sat2prn(msg: object) -> dict:
     """
 
     try:
-        prnmap, _ = id2prnsigmap(msg.identity)
+        prnmap, _ = PRNSIGMAP[str(msg.identity)[0:3]]
 
         sats = {}
         nsat = 0
@@ -339,7 +310,7 @@ def cell2prn(msg: object, sigcode: int = 1) -> dict:
     """
 
     try:
-        prnmap, sigmap = id2prnsigmap(msg.identity)
+        prnmap, sigmap = PRNSIGMAP[str(msg.identity)[0:3]]
 
         sats = []
         nsat = 0
@@ -375,45 +346,6 @@ def cell2prn(msg: object, sigcode: int = 1) -> dict:
         ) from err
 
 
-def id2prnsigmap(ident: str) -> tuple:
-    """
-    Map RTCM3 message identity to MSM satellite PRN and signal ID maps.
-
-    :param str ident: RTCM3 MSM message identity e.g. "1077"
-    :return: tuple of (PRNMAP, SIGMAP)
-    :rtype: tuple
-    :raises: KeyError if ident unknown
-    """
-
-    gnss = RTCM_MSGIDS[ident][0:3]
-    if gnss == "GPS":
-        PRNMAP = GPS_PRN_MAP
-        SIGMAP = GPS_SIG_MAP
-    elif gnss == "GLO":
-        PRNMAP = GLONASS_PRN_MAP
-        SIGMAP = GLONASS_SIG_MAP
-    elif gnss == "GAL":
-        PRNMAP = GALILEO_PRN_MAP
-        SIGMAP = GALILEO_SIG_MAP
-    elif gnss == "SBA":
-        PRNMAP = SBAS_PRN_MAP
-        SIGMAP = SBAS_SIG_MAP
-    elif gnss == "QZS":
-        PRNMAP = QZSS_PRN_MAP
-        SIGMAP = QZSS_SIG_MAP
-    elif gnss == "Bei":
-        PRNMAP = BEIDOU_PRN_MAP
-        SIGMAP = BEIDOU_SIG_MAP
-    elif gnss == "IRN":
-        PRNMAP = IRNSS_PRN_MAP
-        SIGMAP = IRNSS_SIG_MAP
-    else:
-        PRNMAP = None
-        SIGMAP = None
-
-    return (PRNMAP, SIGMAP)
-
-
 def escapeall(val: bytes) -> str:
     """
     Escape all byte characters e.g. b'\\\\x73' rather than b`s`

src/pyrtcm/rtcmmessage.py

@@ -7,6 +7,7 @@
 :copyright: SEMU Consulting © 2022
 :license: BSD 3-Clause
 """
+
 # pylint: disable=invalid-name
 
 import pyrtcm.exceptions as rte
@@ -20,13 +21,12 @@
     crc2bytes,
     escapeall,
     len2bytes,
-    num_setbits,
     sat2prn,
 )
 from pyrtcm.rtcmtypes_core import (
+    ATT_BOOL,
     ATT_NCELL,
     ATT_NSAT,
-    BOOL_GROUPS,
     NCELL,
     NSAT,
     NSIG,
@@ -170,7 +170,7 @@ def _set_attribute_single(
         # one index for each nested level (unless it's a 'boolean' group)
         keyr = key
         for i in index:
-            if i > 0 and keyr not in BOOL_GROUPS:
+            if i > 0 and keyr not in ATT_BOOL:
                 keyr += f"_{i:02d}"
 
         # get value of required number of bits at current payload offset
@@ -192,12 +192,14 @@ def _set_attribute_single(
         # NB: This is predicated on MSM payload dictionaries
         # always having attributes DF394, DF395 and DF396
         # in that order
-        if key == "DF394":  # num of satellites in MSM message
-            setattr(self, NSAT, num_setbits(bitfield))
-        elif key == "DF395":  # num of signals in MSM message
-            setattr(self, NSIG, num_setbits(bitfield))
-        elif key == "DF396":  # num of cells in MSM message
-            setattr(self, NCELL, num_setbits(bitfield))
+        if key in ("DF394", "DF395", "DF396"):
+            n = bin(bitfield).count("1")  # number of bits set
+            if key == "DF394":  # num of satellites in MSM message
+                setattr(self, NSAT, n)
+            elif key == "DF395":  # num of signals in MSM message
+                setattr(self, NSIG, n)
+            elif key == "DF396":  # num of cells in MSM message
+                setattr(self, NCELL, n)
 
         return offset
 

src/pyrtcm/rtcmreader.py

@@ -108,7 +108,7 @@ def read(self) -> tuple:
                 raw_data = None
                 parsed_data = None
                 byte1 = self._read_bytes(1)  # read the first byte
-                # if not rtcm, NMEA or RTCM3, discard and continue
+                # if not UBX, NMEA or RTCM3, discard and continue
                 if byte1 not in (b"\xb5", b"\x24", b"\xd3"):
                     continue
                 byte2 = self._read_bytes(1)
@@ -155,14 +155,13 @@ def _parse_ubx(self, hdr: bytes) -> tuple:
 
         # read the rest of the UBX message from the buffer
         byten = self._read_bytes(4)
-        clsid = byten[0:1]
-        msgid = byten[1:2]
+        msgid = byten[0:2]
         lenb = byten[2:4]
         leni = int.from_bytes(lenb, "little", signed=False)
         byten = self._read_bytes(leni + 2)
         plb = byten[0:leni]
         cksum = byten[leni : leni + 2]
-        raw_data = hdr + clsid + msgid + lenb + plb + cksum
+        raw_data = hdr + msgid + lenb + plb + cksum
         parsed_data = None
         return (raw_data, parsed_data)